Learning Curves for Gaussian Process Regression: Approximations and Bounds

DOI10.1162/089976602753712990MaRDI QIDQ4542429zbMATH OpenOpenAlexDBLPWikidataFDO

Authors Peter Sollich, Anason Halees

Publication date 7 January 2003

Published in Neural Computation (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/cond-mat/0105015

zbMATH Keywords

neural networkslearning curves

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Abstract: We consider the problem of calculating learning curves (i.e., average generalization performance) of Gaussian processes used for regression. On the basis of a simple expression for the generalization error, in terms of the eigenvalue decomposition of the covariance function, we derive a number of approximation schemes. We identify where these become exact, and compare with existing bounds on learning curves; the new approximations, which can be used for any input space dimension, generally get substantially closer to the truth. We also study possible improvements to our approximations. Finally, we use a simple exactly solvable learning scenario to show that there are limits of principle on the quality of approximations and bounds expressible solely in terms of the eigenvalue spectrum of the covariance function.

Recommendations

• scientific article; zbMATH DE number 1849137
• Asymptotic analysis of the learning curve for Gaussian process regression
• Replica theory for learning curves for Gaussian processes on random graphs
• Gaussian processes for machine learning.
• PAC-Bayesian Generalisation Error Bounds for Gaussian Process Classification

Cites work

• Almost optimal differentiation using noisy data
• Finite-size effects in learning and generalization in linear perceptrons
• On average case complexity of linear problems with noisy information

Cited in

(12)

• An uncertainty-aware, mesh-free numerical method for Kolmogorov PDEs
• Can Gaussian Process Regression Be Made Robust Against Model Mismatch?
• Asymptotic generalization error in the online learning of linearized two-layer neural networks
• Replica theory for learning curves for Gaussian processes on random graphs
• Hilbert space methods for reduced-rank Gaussian process regression
• scientific article; zbMATH DE number 6378008 (Why is no real title available?)
• scientific article; zbMATH DE number 6276239 (Why is no real title available?)
• Learning curves for LMS and regular Gaussian processes
• Asymptotic analysis of the learning curve for Gaussian process regression
• Nested aggregation of experts using inducing points for approximated Gaussian process regression
• Learning ``best kernels from data in Gaussian process regression. With application to aerodynamics
• scientific article; zbMATH DE number 1849137 (Why is no real title available?)

This page was built for publication: Learning Curves for Gaussian Process Regression: Approximations and Bounds

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4542429)