Asymptotic Bounds for Smoothness Parameter Estimates in Gaussian Process Interpolation

DOI10.1137/22M149288XzbMath1525.60045arXiv2203.05400OpenAlexW4389047038MaRDI QIDQ6062242

Toni Karvonen

Publication date: 30 November 2023

Published in: SIAM/ASA Journal on Uncertainty Quantification (Search for Journal in Brave)

Abstract: It is common to model a deterministic response function, such as the output of a computer experiment, as a Gaussian process with a Mat'ern covariance kernel. The smoothness parameter of a Mat'ern kernel determines many important properties of the model in the large data limit, including the rate of convergence of the conditional mean to the response function. We prove that the maximum likelihood estimate of the smoothness parameter cannot asymptotically undersmooth the truth when the data are obtained on a fixed bounded subset of $mathbb{R}^d$. That is, if the data-generating response function has Sobolev smoothness $

u_0 + d/2$, then the smoothness parameter estimate cannot be asymptotically less than $ u_0 + d/2$. The lower bound is sharp. Additionally, we show that maximum likelihood estimation finds the "correct" smoothness for a class of compactly supported self-similar functions. We also consider cross-validation and prove an asymptotic lower bound $

u_0$, which however is unlikely to be sharp. The results are based on approximation theory in Sobolev spaces and some general theorems that restrict the set of values that the parameter estimators can take.

Full work available at URL: https://arxiv.org/abs/2203.05400

zbMATH Keywords

parameter estimationGaussian processesreproducing kernel Hilbert spacesfixed-domain asymptoticsMatérn covariance

Mathematics Subject Classification ID

Gaussian processes (60G15) Asymptotic properties of nonparametric inference (62G20) Numerical interpolation (65D05) Hilbert spaces with reproducing kernels (= (proper) functional Hilbert spaces, including de Branges-Rovnyak and other structured spaces) (46E22)

Cites Work

• Bayes procedures for adaptive inference in inverse problems for the white noise model
• A sharp adaptive confidence ball for self-similar functions
• Asymptotic properties of a maximum likelihood estimator with data from a Gaussian process
• Adaptive Bayesian estimation using a Gaussian random field with inverse gamma bandwidth
• Sampling inequalities for infinitely smooth functions, with applications to interpolation and machine learning
• Function spaces in Lipschitz domains and optimal rates of convergence for sampling
• Sobolev error estimates and a Bernstein inequality for scattered data interpolation via radial basis functions
• Estimating the smoothness of a Gaussian random field from irregularly spaced data via higher-order quadratic variations
• Deterministic and stochastic error bounds in numerical analysis
• Interpolation of spatial data. Some theory for kriging
• On nonparametric confidence intervals
• Design and analysis of computer experiments. With comments and a rejoinder by the authors
• Superconvergence of kernel-based interpolation
• An adaptation theory for nonparametric confidence intervals
• Cross validation and maximum likelihood estimations of hyper-parameters of Gaussian processes with model misspecification
• Adaptive confidence interval for pointwise curve estimation.
• Honest adaptive confidence bands and self-similar functions
• Empirical Bayes scaling of Gaussian priors in the white noise model
• Asymptotic behaviour of the empirical Bayes posteriors associated to maximum marginal likelihood estimator
• On fixed-domain asymptotics, parameter estimation and isotropic Gaussian random fields with Matérn covariance functions
• Probabilistic integration: a role in statistical computation?
• Convergence types and rates in generic Karhunen-Loève expansions with applications to sample path properties
• Fixed-domain asymptotics for a subclass of Matérn-type Gaussian random fields
• On the consistent separation of scale and variance for Gaussian random fields
• Confidence bands in density estimation
• Examples of inconsistency in optimization by expected improvement
• Cross-validation estimation of covariance parameters under fixed-domain asymptotics
• An extension of a bound for functions in Sobolev spaces, with applications to \((m, s)\)-spline interpolation and smoothing
• Adaptive nonparametric confidence sets
• Frequentist coverage of adaptive nonparametric Bayesian credible sets
• Approximate interpolation with applications to selecting smoothing parameters
• Regularity of the sample paths of a general second order random field
• Small sample spaces for Gaussian processes
• Stochastic processes with sample paths in reproducing kernel Hilbert spaces
• An Introduction to the Theory of Reproducing Kernel Hilbert Spaces
• Interpolation of spatial data – A stochastic or a deterministic problem?
• Detection of Random Signals in Dependent Gaussian Noise
• Mathematical Foundations of Infinite-Dimensional Statistical Models
• Support Vector Machines
• Posterior consistency for Gaussian process approximations of Bayesian posterior distributions
• Approximation Theory and Algorithms for Data Analysis
• Consistency of empirical Bayes and kernel flow for hierarchical parameter estimation
• Recovery of Sobolev functions restricted to iid sampling
• On the marginal likelihood and cross-validation
• Maximum Likelihood Estimation and Uncertainty Quantification for Gaussian Process Approximation of Deterministic Functions
• On Prediction Properties of Kriging: Uniform Error Bounds and Robustness
• On the Improved Rates of Convergence for Matérn-Type Kernel Ridge Regression with Application to Calibration of Computer Models
• Maximum Likelihood Estimation for a Smooth Gaussian Random Field Model
• Convergence rates of efficient global optimization algorithms
• A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines
• The reproducing kernel Hilbert space structure of the sample paths of a Gaussian process
• Can We Trust Bayesian Uncertainty Quantification from Gaussian Process Priors with Squared Exponential Covariance Kernel?
• Scattered Data Approximation
• Estimating the parameters of some common Gaussian random fields with nugget under fixed-domain asymptotics
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item

This page was built for publication: Asymptotic Bounds for Smoothness Parameter Estimates in Gaussian Process Interpolation