Cross-validation-based adaptive sampling for Gaussian process models
From MaRDI portal
Publication:5862906
DOI10.1137/21M1404260zbMATH Open1493.62473arXiv2005.01814MaRDI QIDQ5862906FDOQ5862906
Authors: Hossein Mohammadi, Daniel B. Williamson, Marc Goodfellow, Peter G. Challenor 
Publication date: 10 March 2022
Published in: SIAM/ASA Journal on Uncertainty Quantification (Search for Journal in Brave)
Abstract: In many real-world applications, we are interested in approximating black-box, costly functions as accurately as possible with the smallest number of function evaluations. A complex computer code is an example of such a function. In this work, a Gaussian process (GP) emulator is used to approximate the output of complex computer code. We consider the problem of extending an initial experiment (set of model runs) sequentially to improve the emulator. A sequential sampling approach based on leave-one-out (LOO) cross-validation is proposed that can be easily extended to a batch mode. This is a desirable property since it saves the user time when parallel computing is available. After fitting a GP to training data points, the expected squared LOO (ES-LOO) error is calculated at each design point. ES-LOO is used as a measure to identify important data points. More precisely, when this quantity is large at a point it means that the quality of prediction depends a great deal on that point and adding more samples nearby could improve the accuracy of the GP. As a result, it is reasonable to select the next sample where ES-LOO is maximised. However, ES-LOO is only known at the experimental design and needs to be estimated at unobserved points. To do this, a second GP is fitted to the ES-LOO errors and where the maximum of the modified expected improvement (EI) criterion occurs is chosen as the next sample. EI is a popular acquisition function in Bayesian optimisation and is used to trade-off between local/global search. However, it has a tendency towards exploitation, meaning that its maximum is close to the (current) "best" sample. To avoid clustering, a modified version of EI, called pseudo expected improvement, is employed which is more explorative than EI yet allows us to discover unexplored regions. Our results show that the proposed sampling method is promising.
Full work available at URL: https://arxiv.org/abs/2005.01814
Recommendations
- Gaussian process hyper-parameter estimation using parallel asymptotically independent Markov sampling
- Computer emulation with nonstationary Gaussian processes
- Exploiting Variance Reduction Potential in Local Gaussian Process Search
- Computer Experiments: Prediction Accuracy, Sample Size and Model Complexity Revisited
- Multiobjective optimization using Gaussian process emulators via stepwise uncertainty reduction
Gaussian processes (60G15) Sequential statistical design (62L05) Inference from stochastic processes and prediction (62M20) Design of statistical experiments (62K99)
Cites Work
- Efficient global optimization of expensive black-box functions
- Bayesian data analysis.
- A survey of cross-validation procedures for model selection
- Multivariate adaptive regression splines
- The design and analysis of computer experiments.
- Design and analysis of computer experiments. With comments and a rejoinder by the authors
- Differential evolution -- a simple and efficient heuristic for global optimization over continuous spaces
- A taxonomy of global optimization methods based on response surfaces
- A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output from a Computer Code
- Convergence properties of the expected improvement algorithm with fixed mean and covariance functions
- Accurate emulators for large-scale computer experiments
- Cross-validation for selecting a model selection procedure
- Title not available (Why is that?)
- Title not available (Why is that?)
- Convergence rates of efficient global optimization algorithms
- Metamodels for computer-based engineering design: Survey and recommendations
- Cross validation and maximum likelihood estimations of hyper-parameters of Gaussian processes with model misspecification
- Near-optimal sensor placements in Gaussian processes: theory, efficient algorithms and empirical studies
- A literature survey of benchmark functions for global optimisation problems
- Title not available (Why is that?)
- Title not available (Why is that?)
- Design of computer experiments: space filling and beyond
- Sequential Design with Mutual Information for Computer Experiments (MICE): Emulation of a Tsunami Model
- A novel hybrid sequential design strategy for global surrogate modeling of computer experiments
- On the convergence rates of expected improvement methods
- Pseudo expected improvement criterion for parallel EGO algorithm
- Quantification of Airfoil Geometry-Induced Aerodynamic Uncertainties---Comparison of Approaches
- Diagnostics-driven nonstationary emulators using kernel mixtures
- Universal Prediction Distribution for Surrogate Models
- Approximate leave-future-out cross-validation for Bayesian time series models
Cited In (6)
- Gradient and uncertainty enhanced sequential sampling for global fit
- Parallel cross-validation: a scalable fitting method for Gaussian process models
- Adaptive Gaussian Process Approximation for Bayesian Inference with Expensive Likelihood Functions
- Gaussian process hyper-parameter estimation using parallel asymptotically independent Markov sampling
- Adaptive multiple importance sampling for Gaussian processes
- Cholesky-based experimental design for Gaussian process and kernel-based emulation and calibration
Uses Software
This page was built for publication: Cross-validation-based adaptive sampling for Gaussian process models
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5862906)
