Uncertainty quantification for sparse spectral variational approximations in Gaussian process regression

DOI10.1214/23-EJS2155arXiv2212.11031MaRDI QIDQ6184877FDOQ6184877

Authors: Dennis Nieman, Botond Szabó, Harry van Zanten 

Publication date: 5 January 2024

Published in: Electronic Journal of Statistics (Search for Journal in Brave)

Abstract: We investigate the frequentist properties of the variational sparse Gaussian Process regression model. In the theoretical analysis we focus on the variational approach with spectral features as inducing variables. We derive guarantees and limitations for the frequentist coverage of the resulting variational credible sets. We also derive sufficient and necessary lower bounds for the number of inducing variables required to achieve minimax posterior contraction rates. The implications of these results are demonstrated for different choices of priors. In a numerical analysis we consider a wider range of inducing variable methods and observe similar phenomena beyond the scope of our theoretical findings.

Full work available at URL: https://arxiv.org/abs/2212.11031

zbMATH Keywords

nonparametric regressionvariational Bayesuncertainty quantificationBayesian asymptoticsinducing variables method

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Nonparametric regression and quantile regression (62G08) Asymptotic properties of nonparametric inference (62G20) Nonparametric tolerance and confidence regions (62G15)

Cites Work

• Asymptotic Statistics
• Gaussian processes for machine learning.
• Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting
• Pattern recognition and machine learning.
• A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines
• Asymptotic methods in statistical decision theory
• Mercer's theorem on general domains: on the interaction between measures, kernels, and RKHSs
• Concentration of tempered posteriors and of their variational approximations
• On the Bernstein-von Mises phenomenon for nonparametric Bayes procedures
• Bayesian inverse problems with Gaussian priors
• An analysis of Bayesian inference for nonparametric regression
• Frequentist coverage of adaptive nonparametric Bayesian credible sets
• Rates of contraction of posterior distributions based on Gaussian process priors
• Title not available (Why is that?)
• Bayesian inference with rescaled Gaussian process priors
• Variational Bayes for High-Dimensional Linear Regression With Sparse Priors
• Asymptotic frequentist coverage properties of Bayesian credible sets for sieve priors
• Adaptive Bayesian inference in the Gaussian sequence model using exponential-variance priors
• Convergence rates of variational posterior distributions
• Convergence of sparse variational inference in Gaussian processes regression
• \(\alpha\)-variational inference with statistical guarantees
• Frequentist consistency of variational Bayes
• Can we trust Bayesian uncertainty quantification from Gaussian process priors with squared exponential covariance kernel?

Cited In (2)

• Sparse pseudo spectral projection methods with directional adaptation for uncertainty quantification
• Emerging directions in Bayesian computation

This page was built for publication: Uncertainty quantification for sparse spectral variational approximations in Gaussian process regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6184877)