Gaussian process regression in the flat limit
From MaRDI portal
Publication:6183872
Abstract: Gaussian process (GP) regression is a fundamental tool in Bayesian statistics. It is also known as kriging and is the Bayesian counterpart to the frequentist kernel ridge regression. Most of the theoretical work on GP regression has focused on a large- asymptotics, characterising the behaviour of GP regression as the amount of data increases. Fixed-sample analysis is much more difficult outside of simple cases, such as locations on a regular grid. In this work we perform a fixed-sample analysis that was first studied in the context of approximation theory by Driscoll & Fornberg (2002), called the "flat limit". In flat-limit asymptotics, the goal is to characterise kernel methods as the length-scale of the kernel function tends to infinity, so that kernels appear flat over the range of the data. Surprisingly, this limit is well-defined, and displays interesting behaviour: Driscoll & Fornberg showed that radial basis interpolation converges in the flat limit to polynomial interpolation, if the kernel is Gaussian. Leveraging recent results on the spectral behaviour of kernel matrices in the flat limit, we study the flat limit of Gaussian process regression. Results show that Gaussian process regression tends in the flat limit to (multivariate) polynomial regression, or (polyharmonic) spline regression, depending on the kernel. Importantly, this holds for both the predictive mean and the predictive variance, so that the posterior predictive distributions become equivalent. Our results have practical consequences: for instance, they show that optimal GP predictions in the sense of leave-one-out loss may occur at very large length-scales, which would be invisible to current implementations because of numerical difficulties.
Recommendations
Cites work
- scientific article; zbMATH DE number 3124366 (Why is no real title available?)
- scientific article; zbMATH DE number 45848 (Why is no real title available?)
- scientific article; zbMATH DE number 3533448 (Why is no real title available?)
- scientific article; zbMATH DE number 1893763 (Why is no real title available?)
- scientific article; zbMATH DE number 5055767 (Why is no real title available?)
- A Stable Algorithm for Flat Radial Basis Functions on a Sphere
- A new look at the statistical model identification
- A taxonomy of global optimization methods based on response surfaces
- An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach
- Bayesian Probabilistic Numerical Methods
- Convergence of Gaussian process regression with estimated hyper-parameters and applications in Bayesian inverse problems
- Determinantal Point Processes Implicitly Regularize Semiparametric Regression Problems
- Determinantal point processes in the flat limit
- Efficient nonparametric Bayesian inference for X-ray transforms
- Expectation Propagation in the Large Data Limit
- Extended L-ensembles: a new representation for determinantal point processes
- From Stein's unbiased risk estimates to the method of generalized cross- validation
- Gaussian Markov Random Fields
- Gaussian processes for machine learning.
- Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter
- Generalized additive models. An introduction with R.
- Interpolation in the limit of increasingly flat radial basis functions
- Interpolation of scattered data: distance matrices and conditionally positive definite functions
- Interpolation of spatial data -- a stochastic or a deterministic problem?
- Interpolation of spatial data. Some theory for kriging
- Linear smoothers and additive models
- Multivariate interpolation at arbitrary points made simple
- Multivariate interpolation by polynomials and radial basis functions
- Multivariate interpolation with increasingly flat radial basis functions of finite smoothness
- Polynomial interpolation in several variables: lattices, differences, and ideals
- Rates of contraction of posterior distributions based on Gaussian process priors
- Scattered Data Approximation
- Semiparametric Regression
- Smoothing by spline functions.
- Spectral properties of kernel matrices in the flat limit
- Stable Computations with Gaussian Radial Basis Functions
- Statistical guarantees for Bayesian uncertainty quantification in nonlinear inverse problems with Gaussian process priors
- The elements of statistical learning. Data mining, inference, and prediction
- Theoretical and computational aspects of multivariate interpolation with increasingly flat radial basis functions
Cited in
(4)
This page was built for publication: Gaussian process regression in the flat limit
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6183872)
