Adaptive Bayesian density regression for high-dimensional data

DOI10.3150/14-BEJ663MaRDI QIDQ5963506zbMATH OpenOpenAlexWikidataFDO

Authors Weining Shen, Subhashis Ghosal

Publication date 22 February 2016

Published in Bernoulli (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1403.2695, https://projecteuclid.org/euclid.bj/1443620855

zbMATH Keywords

variable selectionhigh-dimensional modelsnonparametric Bayesian inferenceadaptive estimationposterior contraction ratedensity regressionMCMC-free computation

Mathematics Subject Classification ID

Bayesian inference (62F15) Density estimation (62G07) Nonparametric regression and quantile regression (62G08) Asymptotic properties of nonparametric inference (62G20)

Abstract: Density regression provides a flexible strategy for modeling the distribution of a response variable $Y$ given predictors $mathbfX=(X_1,ldots,X_p)$ by letting that the conditional density of $Y$ given $mathbfX$ as a completely unknown function and allowing its shape to change with the value of $mathbfX$. The number of predictors $p$ may be very large, possibly much larger than the number of observations $n$, but the conditional density is assumed to depend only on a much smaller number of predictors, which are unknown. In addition to estimation, the goal is also to select the important predictors which actually affect the true conditional density. We consider a nonparametric Bayesian approach to density regression by constructing a random series prior based on tensor products of spline functions. The proposed prior also incorporates the issue of variable selection. We show that the posterior distribution of the conditional density contracts adaptively at the truth nearly at the optimal oracle rate, determined by the unknown sparsity and smoothness levels, even in the ultra high-dimensional settings where $p$ increases exponentially with $n$. The result is also extended to the anisotropic case where the degree of smoothness can vary in different directions, and both random and deterministic predictors are considered. We also propose a technique to calculate posterior moments of the conditional density function without requiring Markov chain Monte Carlo methods.

Recommendations

• Bayesian Density Regression
• Regression density estimation using smooth adaptive Gaussian mixtures
• Adaptive Bayesian estimation of conditional densities
• Empirical Bayes conditional density estimation
• Bayesian isotonic density regression

Cites work

• scientific article; zbMATH DE number 5190601 (Why is no real title available?)
• scientific article; zbMATH DE number 1034042 (Why is no real title available?)
• scientific article; zbMATH DE number 2135362 (Why is no real title available?)
• scientific article; zbMATH DE number 795289 (Why is no real title available?)
• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• A class of mixtures of dependent tail-free processes
• A crossvalidation method for estimating conditional densities
• A practical guide to splines.
• Adaptive Bayesian multivariate density estimation with Dirichlet mixtures
• Adaptive Bayesian procedures using random series priors
• Adaptive estimation of multivariate functions using conditionally Gaussian tensor-product spline priors
• Bayes Model Averaging with Selection of Regressors
• Bayesian Density Regression
• Bayesian curve fitting using multivariate normal mixtures
• Bayesian density regression with logistic Gaussian process and subspace projection
• Bayesian variable selection for high dimensional generalized linear models: convergence rates of the fitted densities
• Convergence rates of posterior distributions for non iid observations
• Convergence rates of posterior distributions.
• Estimation of conditional densities and sensitivity measures in nonlinear dynamical systems
• High-dimensional generalized linear models and the lasso
• Kernel stick-breaking processes
• Methods for Estimating a Conditional Distribution Function
• Mirror averaging with sparsity priors
• Multivariate Bayesian Variable Selection and Prediction
• Multivariate density estimation under sup-norm loss: oracle approach, adaptation and independence structure
• Needles and straw in a haystack: posterior concentration for possibly sparse sequences
• Nonparametric Bayes conditional distribution modeling with variable selection
• On estimating a density using Hellinger distance and some other strange facts
• Order-Based Dependent Dirichlet Processes
• Persistene in high-dimensional linear predictor-selection and the virtue of overparametrization
• Posterior concentration rates for infinite dimensional exponential families
• Posterior consistency in conditional density estimation by covariate dependent mixtures
• Posterior consistency in conditional distribution estimation
• Posterior convergence rates of Dirichlet mixtures at smooth densities
• Rates of contraction of posterior distributions based on Gaussian process priors
• Rates of convergence of posterior distributions.
• Reversible jump Markov chain Monte Carlo computation and Bayesian model determination
• Statistics for high-dimensional data. Methods, theory and applications.
• The multivariate beta process and an extension of the Pólya tree model
• Variable selection in nonparametric additive models

Cited in

(18)

• Tractable Bayesian density regression via logit stick-breaking priors
• Nonparametric Bayes conditional distribution modeling with variable selection
• Discovering interactions using covariate informed random partition models
• Regression density estimation using smooth adaptive Gaussian mixtures
• Contraction properties of shrinkage priors in logistic regression
• Supremum norm posterior contraction and credible sets for nonparametric multivariate regression
• Empirical Bayes conditional density estimation
• Adaptive Bayesian estimation of conditional densities
• Scalable Bayesian nonparametric regression via a Plackett-Luce model for conditional ranks
• Converting high-dimensional regression to high-dimensional conditional density estimation
• Bayesian inference in high-dimensional linear models using an empirical correlation-adaptive prior
• Adaptive Bayesian estimation of conditional discrete-continuous distributions with an application to stock market trading activity
• Contraction of a quasi-Bayesian model with shrinkage priors in precision matrix estimation
• Nonparametric estimation of conditional densities by generalized random forests
• Simultaneous variable selection and component selection for regression density estimation with mixtures of heteroscedastic experts
• Nonparametric density estimation on complex domains using manifold-aware Bayesian additive tree models
• Posterior contraction rates of density derivative estimation
• Bayesian density regression with logistic Gaussian process and subspace projection

This page was built for publication: Adaptive Bayesian density regression for high-dimensional data

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5963506)