Variable selection consistency of Gaussian process regression
From MaRDI portal
Abstract: Bayesian nonparametric regression under a rescaled Gaussian process prior offers smoothness-adaptive function estimation with near minimax-optimal error rates. Hierarchical extensions of this approach, equipped with stochastic variable selection, are known to also adapt to the unknown intrinsic dimension of a sparse true regression function. But it remains unclear if such extensions offer variable selection consistency, i.e., if the true subset of important variables could be consistently learned from the data. It is shown here that variable consistency may indeed be achieved with such models at least when the true regression function has finite smoothness to induce a polynomially larger penalty on inclusion of false positive predictors. Our result covers the high dimensional asymptotic setting where the predictor dimension is allowed to grow with the sample size. The proof utilizes Schwartz theory to establish that the posterior probability of wrong selection vanishes asymptotically. A necessary and challenging technical development involves providing sharp upper and lower bounds to small ball probabilities at all rescaling levels of the Gaussian process prior, a result that could be of independent interest.
Recommendations
- Bayesian variable selection with shrinking and diffusing priors
- Variable selection for nonparametric Gaussian process priors: Models and computational strategies
- On Model Selection Consistency of Bayesian Method for Normal Linear Models
- High-dimensional posterior consistency for hierarchical non-local priors in regression
- Posterior model consistency in variable selection as the model dimension grows
Cites work
- Adaptive Bayesian estimation using a Gaussian random field with inverse gamma bandwidth
- Adaptive Bernstein-von Mises theorems in Gaussian white noise
- Adaptive variable selection in nonparametric sparse additive models
- Anisotropic function estimation using multi-bandwidth Gaussian processes
- Approximation, metric entropy and small ball estimates for Gaussian measures
- Bayesian linear regression with sparse priors
- Bayesian variable selection with shrinking and diffusing priors
- Confidence bands in density estimation
- Convergence rates of posterior distributions for non iid observations
- Convergence rates of posterior distributions.
- Decoupling shrinkage and selection in Bayesian linear models: a posterior summary perspective
- Frequentist coverage of adaptive nonparametric Bayesian credible sets
- Fundamentals of nonparametric Bayesian inference
- Gaussian processes for machine learning.
- High-dimensional statistics. A non-asymptotic viewpoint
- Honest adaptive confidence bands and self-similar functions
- Information rates of nonparametric Gaussian process methods
- Information-Theoretic Limits on Sparsity Recovery in the High-Dimensional and Noisy Setting
- Lower bounds for posterior rates with Gaussian process priors
- Metric entropy and the small ball problem for Gaussian measures
- Minimax risks for sparse regressions: ultra-high dimensional phenomenons
- Minimax-optimal nonparametric regression in high dimensions
- Minimax-optimal rates for sparse additive models over kernel classes via convex programming
- Nonparametric Bayesian model selection and averaging
- On the computational complexity of high-dimensional Bayesian variable selection
- Optimal global rates of convergence for nonparametric regression
- Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh
- Rodeo: Sparse, greedy nonparametric regression
- SLOPE-adaptive variable selection via convex optimization
- Selection of variables and dimension reduction in high-dimensional non-parametric regression
- Small Deviations of Smooth Stationary Gaussian Processes
- Statistics for high-dimensional data. Methods, theory and applications.
- Tight conditions for consistency of variable selection in the context of high dimensionality
- Variable selection in nonparametric regression with continuous covariates
Cited in
(14)- Generalized Variable Selection Algorithms for Gaussian Process Models by LASSO-Like Penalty
- Bayesian regression based on principal components for high-dimensional data
- Variable selection for nonparametric Gaussian process priors: Models and computational strategies
- Contraction rates and projection subspace estimation with Gaussian process priors in high dimension
- Adaptive Bayesian regression on data with low intrinsic dimensionality
- Additive Multi-Index Gaussian Process Modeling, with Application to Multi-Physics Surrogate Modeling of the Quark-Gluon Plasma
- On the Consistency of Bayesian Variable Selection for High Dimensional Binary Regression and Classification
- Adaptive variational Bayes: optimality, computation and applications
- Bayesian adaptive variable selection with a generalized g-prior
- Deep horseshoe Gaussian processes
- Optimal Bayesian estimation of Gaussian mixtures with growing number of components
- High-dimensional posterior consistency for hierarchical non-local priors in regression
- Posterior concentration for Gaussian process priors under rescaled and hierarchical Matérn and confluent hypergeometric covariance functions
- Streaming-Data Selection for Gaussian-Process Modelling
This page was built for publication: Variable selection consistency of Gaussian process regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2054515)
