Bayesian manifold regression

DOI10.1214/15-AOS1390MaRDI QIDQ282481zbMATH OpenOpenAlexFDO

Authors Yun Yang, David Dunson

Publication date 12 May 2016

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1305.0617, https://projecteuclid.org/euclid.aos/1458245738

zbMATH Keywords

asymptoticscontraction ratesdimensionality reductionGaussian processmanifold learningnonparametric Bayessubspace learning

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Nonparametric inference (62G99) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05)

Abstract: There is increasing interest in the problem of nonparametric regression with high-dimensional predictors. When the number of predictors $D$ is large, one encounters a daunting problem in attempting to estimate a $D$-dimensional surface based on limited data. Fortunately, in many applications, the support of the data is concentrated on a $d$-dimensional subspace with $dllD$. Manifold learning attempts to estimate this subspace. Our focus is on developing computationally tractable and theoretically supported Bayesian nonparametric regression methods in this context. When the subspace corresponds to a locally-Euclidean compact Riemannian manifold, we show that a Gaussian process regression approach can be applied that leads to the minimax optimal adaptive rate in estimating the regression function under some conditions. The proposed model bypasses the need to estimate the manifold, and can be implemented using standard algorithms for posterior computation in Gaussian processes. Finite sample performance is illustrated in an example data analysis.

Recommendations

• Compressed Gaussian process for manifold regression
• Multivariate Bayesian function estimation
• Minimax-optimal nonparametric regression in high dimensions
• Local linear regression on manifolds and its geometric interpretation
• A spectral series approach to high-dimensional nonparametric regression

Cites work

• A tree-based regressor that adapts to intrinsic dimension
• Adaptive Bayesian estimation using a Gaussian random field with inverse gamma bandwidth
• Anisotropic function estimation using multi-bandwidth Gaussian processes
• Bayesian density regression with logistic Gaussian process and subspace projection
• Bayesian monotone regression using Gaussian process projection
• Classification via Bayesian Nonparametric Learning of Affine Subspaces
• Compressive Sensing on Manifolds Using a Nonparametric Mixture of Factor Analyzers: Algorithm and Performance Bounds
• Convergence rates of posterior distributions for non iid observations
• Convergence rates of posterior distributions.
• scientific article; zbMATH DE number 5654889 (Why is no real title available?)
• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• Information rates of nonparametric Gaussian process methods
• Laplacian Eigenmaps for Dimensionality Reduction and Data Representation
• Latent factor models for density estimation
• Learning and approximation by Gaussians on Riemannian manifolds
• On Local Intrinsic Dimension Estimation and Its Applications
• Optimal global rates of convergence for nonparametric regression
• Rates of contraction for posterior distributions in \(L^{r}\)-metrics, \(1 \leq r \leq \infty\)
• Regularization and Variable Selection Via the Elastic Net
• Sufficient dimension reduction via Bayesian mixture modeling
• The semiparametric Bernstein-von Mises theorem
• Theory of Reproducing Kernels
• Thomas Bayes' walk on manifolds
• Universal algorithms for learning theory. I: Piecewise constant functions.
• Universal algorithms for learning theory. II: Piecewise polynomial functions
• Variable selection for nonparametric Gaussian process priors: Models and computational strategies

Cited in

(81)

• High-dimensional intrinsic interpolation using Gaussian process regression and diffusion maps
• Flexible Bayesian dynamic modeling of correlation and covariance matrices
• Probabilistic modelling of general noisy multi-manifold data sets
• Adaptive learning rates for support vector machines working on data with low intrinsic dimension
• Semi-parametric Bayes regression with network-valued covariates
• Bandit and covariate processes, with finite or non-denumerable set of arms
• Kernel-based system identification with manifold regularization: a Bayesian perspective
• Airflow recovery from thoracic and abdominal movements using synchrosqueezing transform and locally stationary Gaussian process regression
• Multi-output regression on the output manifold
• Longitudinal image analysis via path regression on the image manifold
• Extrinsic Gaussian processes for regression and classification on manifolds
• Probabilistic integration: a role in statistical computation?
• Minimax-optimal nonparametric regression in high dimensions
• Anisotropic function estimation using multi-bandwidth Gaussian processes
• Diffusion \(K\)-means clustering on manifolds: provable exact recovery via semidefinite relaxations
• Estimation of a regression function on a manifold by fully connected deep neural networks
• Compressed Gaussian process for manifold regression
• Regression on manifolds using data-dependent regularization with applications in computer vision
• scientific article; zbMATH DE number 7387620 (Why is no real title available?)
• scientific article; zbMATH DE number 7387621 (Why is no real title available?)
• Gaussian process landmarking on manifolds
• scientific article; zbMATH DE number 7625193 (Why is no real title available?)
• Symmetry tests for manifold-valued random variables
• Intrinsic dimension adaptive partitioning for kernel methods
• Adaptive approximation and generalization of deep neural network with intrinsic dimensionality
• Variational Bayesian data analysis on manifold
• Local linear regression on manifolds and its geometric interpretation
• Functional regression on the manifold with contamination
• Can we trust Bayesian uncertainty quantification from Gaussian process priors with squared exponential covariance kernel?
• Gaussian Process Subspace Prediction for Model Reduction
• A spectral series approach to high-dimensional nonparametric regression
• Rates of the strong uniform consistency for the kernel-type regression function estimators with general kernels on manifolds
• Sets that maximize probability and a related variational problem
• Deep nonparametric regression on approximate manifolds: nonasymptotic error bounds with polynomial prefactors
• Minimax rate of distribution estimation on unknown submanifolds under adversarial losses
• A diffusion process perspective on posterior contraction rates for parameters
• Solving PDEs on spheres with physics-informed convolutional neural networks
• Contraction rates and projection subspace estimation with Gaussian process priors in high dimension
• Generalized fiducial inference on differentiable manifolds
• Deep horseshoe Gaussian processes
• Review of large-scale simulation optimization
• Multivariate Gaussian RBF-net for smooth function estimation and variable selection
• A functional linear regression for high-resolution 3D faces
• Deep neural networks for nonparametric interaction models with diverging dimension
• Adaptive Bayesian regression on data with low intrinsic dimensionality
• Fifty years of stochastic simulation: where we are and where we need to go
• Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Discussion of: ``Statistical exploration of the manifold hypothesis
• Reply to the discussion of: ``Statistical exploration of the manifold hypothesis
• Scalable Bayesian inference for heat kernel Gaussian processes on manifolds
• Double soft-thresholded model for multi-group scalar on vector-valued image regression

Describes a project that uses

Uses Software

• COIL-100

This page was built for publication: Bayesian manifold regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q282481)