Adaptive Bayesian estimation using a Gaussian random field with inverse gamma bandwidth

DOI10.1214/08-AOS678zbMATH Open1173.62021arXiv0908.3556OpenAlexW2157596407MaRDI QIDQ834358FDOQ834358

Authors: J. H. van Zanten, Aad van der Vaart 

Publication date: 19 August 2009

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: We consider nonparametric Bayesian estimation inference using a rescaled smooth Gaussian field as a prior for a multidimensional function. The rescaling is achieved using a Gamma variable and the procedure can be viewed as choosing an inverse Gamma bandwidth. The procedure is studied from a frequentist perspective in three statistical settings involving replicated observations (density estimation, regression and classification). We prove that the resulting posterior distribution shrinks to the distribution that generates the data at a speed which is minimax-optimal up to a logarithmic factor, whatever the regularity level of the data-generating distribution. Thus the hierachical Bayesian procedure, with a fixed prior, is shown to be fully adaptive.

Full work available at URL: https://arxiv.org/abs/0908.3556

Recommendations

• Adaptive nonparametric Bayesian inference using location-scale mixture priors
• Bayesian adaptation
• Adaptive Bayesian estimation in Gaussian sequence space models
• Adaptive Bayesian procedures using random series priors
• Adaptive Bayesian density estimation in \(L^p\)-metrics with Pitman-Yor or normalized inverse-Gaussian process kernel mixtures

zbMATH Keywords

Bayesian inferenceclassificationnonparametric regressionrate of convergenceadaptationnonparametric density estimationposterior distributionGaussian process priors

Mathematics Subject Classification ID

Bayesian inference (62F15) Density estimation (62G07) Nonparametric regression and quantile regression (62G08) Asymptotic properties of nonparametric inference (62G20) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Random fields; image analysis (62M40)

Cites Work

• Weak convergence and empirical processes. With applications to statistics
• The Logistic Normal Distribution for Bayesian, Nonparametric, Predictive Densities
• Title not available (Why is that?)
• An asymptotically optimal window selection rule for kernel density estimates
• Adaptive wavelet estimation: A block thresholding and oracle inequality approach
• Gaussian processes for machine learning.
• Title not available (Why is that?)
• Density estimation by wavelet thresholding
• Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities.
• On posterior consistency in nonparametric regression problems
• Towards a practicable Bayesian nonparametric density estimator
• Title not available (Why is that?)
• A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines
• Convergence rates of posterior distributions.
• Nonparametric curve estimation. Methods, theory, and applications
• Risk bounds for model selection via penalization
• Asymptotically minimax adaptive estimation. II: Schemes without optimal adaption. Adaptive estimates
• Title not available (Why is that?)
• Asymptotically Minimax Adaptive Estimation. I: Upper Bounds. Optimally Adaptive Estimates
• Minimum complexity density estimation
• Title not available (Why is that?)
• Nonparametric Bayesian model selection and averaging
• Title not available (Why is that?)
• On universal Bayesian adaptation
• Rates of contraction of posterior distributions based on Gaussian process priors
• Posterior convergence rates of Dirichlet mixtures at smooth densities
• Posterior consistency of logistic Gaussian process priors in density estimation
• Analytic Extensions of Differentiable Functions Defined in Closed Sets
• Title not available (Why is that?)
• Nonparametric binary regression using a Gaussian process prior
• Title not available (Why is that?)
• Posterior consistency of Gaussian process prior for nonparametric binary regression
• Title not available (Why is that?)
• The Gaussian measure of shifted balls
• Asymptotically efficient estimation for analytic distributions
• Spline smoothing in regression models and asymptotic efficiency in \(L_ 2\)
• Metric entropy and the small ball problem for Gaussian measures
• Convergence rates for posterior distributions and adaptive estimation
• Approximation, metric entropy and small ball estimates for Gaussian measures
• Adaptive minimax estimation of infinitely differentiable functions.
• Bayesian inference with rescaled Gaussian process priors
• Introduction to probability and measure
• Title not available (Why is that?)
• Learning algorithm for nonparametric filtering
• Adaptive asymptotically minimax estimators of smooth signals
• Alternative posterior consistency results in nonparametric binary regression using Gaussian process priors
• Asymptotically local minimax estimation of infinitely smooth density with censored data

Cited In (81)

• Adaptive rates of contraction of posterior distributions in Bayesian wavelet regression
• Extrinsic Gaussian processes for regression and classification on manifolds
• Bayesian fractional posteriors
• Bayesian inverse problems with non-conjugate priors
• Bayesian closed surface fitting through tensor products
• Bayes procedures for adaptive inference in inverse problems for the white noise model
• On posterior distribution of Bayesian wavelet thresholding
• Nonparametric Bayesian inference for ergodic diffusions
• A closer look at covering number bounds for Gaussian kernels
• Constructing Priors that Penalize the Complexity of Gaussian Random Fields
• Adaptive nonparametric Bayesian inference using location-scale mixture priors
• Bayesian nonparametric regression with varying residual density
• Consistency of posterior distributions for heteroscedastic nonparametric regression models
• Bayesian manifold regression
• Rates of convergence for the posterior distributions of mixtures of betas and adaptive nonparametric estimation of the density
• Anisotropic function estimation using multi-bandwidth Gaussian processes
• Rates of contraction for posterior distributions in \(L^{r}\)-metrics, \(1 \leq r \leq \infty\)
• Consistency of Bayesian inference with Gaussian process priors in an elliptic inverse problem
• Rate exact Bayesian adaptation with modified block priors
• Minimax-optimal nonparametric regression in high dimensions
• On adaptive posterior concentration rates
• Posterior consistency in conditional density estimation by covariate dependent mixtures
• Adaptive Bayesian estimation of conditional densities
• Oracle posterior contraction rates under hierarchical priors
• Empirical Bayes scaling of Gaussian priors in the white noise model
• Adaptive Bayesian density estimation in \(L^p\)-metrics with Pitman-Yor or normalized inverse-Gaussian process kernel mixtures
• Posterior convergence for Bayesian functional linear regression
• A geometric variational approach to Bayesian inference
• Rates for Bayesian estimation of location-scale mixtures of super-smooth densities
• Bayesian fixed-domain asymptotics for covariance parameters in a Gaussian process model
• Adaptive estimation of stationary Gaussian fields
• Lower bound for the oracle projection posterior convergence rate
• Adaptive Bayesian inference in the Gaussian sequence model using exponential-variance priors
• Adaptive Bayesian credible sets in regression with a Gaussian process prior
• Bayesian adaptation
• Semiparametric Bayesian causal inference
• Variable selection consistency of Gaussian process regression
• Flexible Bayesian dynamic modeling of correlation and covariance matrices
• Adaptive Bayesian procedures using random series priors
• Bayesian optimal adaptive estimation using a sieve prior
• Spike and slab Pólya tree posterior densities: adaptive inference
• Adaptive estimation of multivariate functions using conditionally Gaussian tensor-product spline priors
• Semiparametric Bernstein-von Mises for the error standard deviation
• Posterior contraction rates of density derivative estimation
• Consistent online Gaussian process regression without the sample complexity bottleneck
• A note on the smoothness of densities
• Thomas Bayes' walk on manifolds
• Posterior consistency in conditional distribution estimation
• Rates of contraction of posterior distributions based on \(p\)-exponential priors
• Adaptive learning rates for support vector machines working on data with low intrinsic dimension
• Laplace approximation for logistic Gaussian process density estimation and regression
• Bayesian Projected Calibration of Computer Models
• Full adaptation to smoothness using randomly truncated series priors with Gaussian coefficients and inverse gamma scaling
• Hierarchical Bayesian level set inversion
• Gaussian processes for history-matching: application to an unconventional gas reservoir
• Nonparametric Bernstein-von Mises theorems in Gaussian white noise
• Learning rates for kernel-based expectile regression
• Regularizing priors for linear inverse problems
• Bayesian regression with nonparametric heteroskedasticity
• A theoretical framework of the scaled Gaussian stochastic process in prediction and calibration
• Adaptive Bayesian nonparametric regression using a kernel mixture of polynomials with application to partial linear models
• Posterior contraction in Gaussian process regression using Wasserstein approximations
• Rates of contraction with respect to \(L_2\)-distance for Bayesian nonparametric regression
• On some aspects of the asymptotic properties of Bayesian approaches in nonparametric and semiparametric models
• Heavy-Tailed Density Estimation
• A Bayesian approach for consistent reconstruction of inclusions
• Comparing and Weighting Imperfect Models Using D-Probabilities
• Adaptive Bayesian estimation in Gaussian sequence space models
• Recovering gradients from sparsely observed functional data
• Adaptive method for indirect identification of the statistical properties of random fields in a Bayesian framework
• Asymptotic Bounds for Smoothness Parameter Estimates in Gaussian Process Interpolation
• Can we trust Bayesian uncertainty quantification from Gaussian process priors with squared exponential covariance kernel?
• Wasserstein convergence in Bayesian and frequentist deconvolution models
• Heavy-tailed Bayesian nonparametric adaptation
• Comparison theorems on large-margin learning
• The Bayesian approach to inverse Robin problems
• Discussion of ``Frequentist coverage of adaptive nonparametric Bayesian credible sets
• Optimal learning with Gaussians and correntropy loss
• Bayesian estimation of nonlinear Hawkes processes
• A nonparametric Bayesian methodology for regression discontinuity designs
• Data-driven priors and their posterior concentration rates

This page was built for publication: Adaptive Bayesian estimation using a Gaussian random field with inverse gamma bandwidth

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q834358)