Why not to use the Gaussian kernel

DOI10.48550/arXiv.2608.26974MaRDI QIDQ7285091arXivFDO

Authors Toni Karvonen, Chris J. Oates

Publication date 27 August 2026

Published in arXiv

Tldr This paper argues against using the Gaussian kernel in Gaussian process regression and classification, showing it can lead to overconfidence and numerical instability. We show its analyticity leads to unrealistic conditional variance and ill-conditioning.

Full work available at URL https://doi.org/10.48550/arXiv.2608.26974

arXiv classification

cs.LGmath.NAmath.STstat.ML

Cites work

• 2010 Rietz lecture: When does the screening effect hold?
• A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines
• An introduction to the theory of reproducing kernel Hilbert spaces
• Approximation in Hilbert spaces of the Gaussian and related analytic kernels
• Cases for the nugget in modeling computer experiments
• Comparing scale parameter estimators for Gaussian process interpolation with the Brownian motion prior: leave-one-out cross validation and maximum likelihood
• Conditioning of the stationary Kriging matrices for some well-known covariance models
• Convergence properties of the expected improvement algorithm with fixed mean and covariance functions
• Design and analysis of computer experiments
• Error estimates and condition numbers for radial basis function interpolation
• Examples of inconsistency in optimization by expected improvement
• Frequentist coverage of adaptive nonparametric Bayesian credible sets
• Gaussian Processes for Machine Learning
• Hilbert space methods for reduced-rank Gaussian process regression
• How fast do radial basis function interpolants of analytic functions converge?
• Improved estimates for condition numbers of radial basis function interpolation matrices
• Kernel Mean Embedding of Distributions: A Review and Beyond
• Kernel techniques: From machine learning to meshless methods
• Lévy flights: Exact results and asymptotics beyond all orders
• Maximum likelihood estimation and uncertainty quantification for Gaussian process approximation of deterministic functions
• On non-inclusion of certain functions in reproducing kernel Hilbert spaces
• On the condition number of covariance matrices in kriging, estimation, and simulation of random fields
• Optimal regression rates for SVMs using Gaussian kernels
• Orthonormal expansions for translation-invariant kernels
• Pointwise consistency of the kriging predictor with known mean and covariance functions
• Reproducing kernel Hilbert spaces associated with analytic translation-invariant Mercer kernels
• Reproducing kernels of Sobolev spaces on \(\mathbb{R}^d\) and applications to embedding constants and tractability
• Robust Gaussian stochastic process emulation
• Sampling inequalities for infinitely smooth functions, with applications to interpolation and machine learning
• Scale estimation and rate-unbiasedness for Gaussian processes under smoothness misspecification
• Scattered Data Approximation
• Scikit-learn: machine learning in Python
• Some properties of Gaussian reproducing kernel Hilbert spaces and their implications for function approximation and learning theory
• Sparse spectrum Gaussian process regression
• Spectral gaps for sets and measures
• Support Vector Machines
• The screening effect in kriging
• The zero set of a real analytic function
• Univariate interpolation by exponential functions and Gaussian RBFs for generic sets of nodes
• Universal kernels
• Wendland functions with increasing smoothness converge to a Gaussian

This page was built for publication: Why not to use the Gaussian kernel

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q7285091)