Bayesian linear regression with sparse priors

DOI10.1214/15-AOS1334zbMATH Open1486.62197arXiv1403.0735WikidataQ60142486 ScholiaQ60142486MaRDI QIDQ888501FDOQ888501

Authors: Ismaël Castillo, Johannes Schmidt-Hieber, Aad van der Vaart 

Publication date: 30 October 2015

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: We study full Bayesian procedures for high-dimensional linear regression under sparsity constraints. The prior is a mixture of point masses at zero and continuous distributions. Under compatibility conditions on the design matrix, the posterior distribution is shown to contract at the optimal rate for recovery of the unknown sparse vector, and to give optimal prediction of the response vector. It is also shown to select the correct sparse model, or at least the coefficients that are significantly different from zero. The asymptotic shape of the posterior distribution is characterized and employed to the construction and study of credible sets for uncertainty quantification.

Full work available at URL: https://arxiv.org/abs/1403.0735

Recommendations

• Bayesian sparse linear regression with unknown symmetric error
• Finite sample posterior concentration in high-dimensional regression
• Empirical Bayes posterior concentration in sparse high-dimensional linear models
• Posterior consistency in linear models under shrinkage priors
• Unified Bayesian theory of sparse linear regression with nuisance parameters

zbMATH Keywords

Bayesian inferencesparsity

Mathematics Subject Classification ID

Bayesian inference (62F15) Nonparametric estimation (62G05) Asymptotic properties of nonparametric inference (62G20) Linear regression; mixed models (62J05)

Cites Work

• Nearly unbiased variable selection under minimax concave penalty
• EMVS: the EM approach to Bayesian variable selection
• Correlated variables in regression: clustering and sparse estimation
• Title not available (Why is that?)
• Statistics for high-dimensional data. Methods, theory and applications.
• On the conditions used to prove oracle results for the Lasso
• Simultaneous analysis of Lasso and Dantzig selector
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• Inference with normal-gamma prior distributions in regression problems
• Title not available (Why is that?)
• The horseshoe estimator for sparse signals
• Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
• Sparsity oracle inequalities for the Lasso
• Adapting to unknown sparsity by controlling the false discovery rate
• Bayesian Variable Selection in Linear Regression
• Shotgun Stochastic Search for “Largep” Regression
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• The sparsity and bias of the LASSO selection in high-dimensional linear regression
• Convergence rates of posterior distributions.
• Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion
• Spike and slab variable selection: frequentist and Bayesian strategies
• Calibration and empirical Bayes variable selection
• Stable recovery of sparse overcomplete representations in the presence of noise
• On Bayesian model and variable selection using MCMC
• Efficient Empirical Bayes Variable Selection and Estimation in Linear Models
• Evolutionary stochastic search for Bayesian model exploration
• Information Theory and Mixing Least-Squares Regressions
• Bayesian lasso regression
• Limiting laws of coherence of random matrices with applications to testing covariance structure and construction of compressed sensing matrices
• Oracle convergence rate of posterior under projection prior and Bayesian model selection
• Exponential screening and optimal rates of sparse estimation
• Needles and straw in a haystack: posterior concentration for possibly sparse sequences
• Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators
• Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
• Bayesian linear regression with sparse priors
• On higher order isotropy conditions and lower bounds for sparse quadratic forms
• The Variable Selection Problem
• Sequential Monte Carlo on large binary sampling spaces
• On variational Bayes estimation and variational information criteria for linear regression models
• Bayesian variable selection in structured high-dimensional covariate spaces with applications in genomics
• Estimation and variable selection with exponential weights
• Sparse estimation by exponential weighting
• Aggregation by Exponential Weighting and Sharp Oracle Inequalities
• Bayesian variable selection via particle stochastic search
• MAP model selection in Gaussian regression

Cited In (only showing first 100 items - show all)

• Neuronized Priors for Bayesian Sparse Linear Regression
• Strong selection consistency of Bayesian vector autoregressive models based on a pseudo-likelihood approach
• Variable Selection in the Presence of Factors: A Model Selection Perspective
• Penalising model component complexity: a principled, practical approach to constructing priors
• Needles and straw in a haystack: robust confidence for possibly sparse sequences
• The horseshoe estimator: posterior concentration around nearly black vectors
• Bayesian sparse linear regression with unknown symmetric error
• Convergence rates of variational posterior distributions
• Spike-and-slab Lasso biclustering
• Fast Bayesian variable selection for high dimensional linear models: marginal solo spike and slab priors
• Empirical Bayes oracle uncertainty quantification for regression
• Variational Bayes for High-Dimensional Linear Regression With Sparse Priors
• Conditions for posterior contraction in the sparse normal means problem
• On Bayesian oracle properties
• Sub-optimality of some continuous shrinkage priors
• Bayesian effect fusion for categorical predictors
• Bayesian estimation of sparse signals with a continuous spike-and-slab prior
• A Bayesian nonparametric multiple testing procedure for comparing several treatments against a control
• Prior distributions for objective Bayesian analysis
• The spike-and-slab LASSO
• Criteria for posterior consistency and convergence at a rate
• Variational nonparametric discriminant analysis
• Well-posed Bayesian inverse problems with infinitely divisible and heavy-tailed prior measures
• Maximum pairwise Bayes factors for covariance structure testing
• Nonparametric Bayesian analysis of the compound Poisson prior for support boundary recovery
• Bayesian linear regression with sparse priors
• On frequentist coverage errors of Bayesian credible sets in moderately high dimensions
• Posterior concentration for Bayesian regression trees and forests
• Sharp oracle inequalities for low-complexity priors
• Quasi-Bayesian estimation of large Gaussian graphical models
• Bayesian factor-adjusted sparse regression
• Bayesian MIDAS penalized regressions: estimation, selection, and prediction
• Consistent Bayesian sparsity selection for high-dimensional Gaussian DAG models with multiplicative and beta-mixture priors
• Bernstein-von Mises theorems for functionals of the covariance matrix
• Bayesian linear regression for multivariate responses under group sparsity
• A Critical Review of LASSO and Its Derivatives for Variable Selection Under Dependence Among Covariates
• Oracle posterior contraction rates under hierarchical priors
• Bayesian Estimation of Gaussian Conditional Random Fields
• Bayesian bridge regression
• Needles and straw in a haystack: posterior concentration for possibly sparse sequences
• A general framework for Bayes structured linear models
• Bayesian inference for high-dimensional decomposable graphs
• Unified Bayesian theory of sparse linear regression with nuisance parameters
• Sparse Bayesian inference with regularized Gaussian distributions ^*
• Minimax posterior convergence rates and model selection consistency in high-dimensional DAG models based on sparse Cholesky factors
• Lasso meets horseshoe: a survey
• On the beta prime prior for scale parameters in high-dimensional Bayesian regression models
• Empirical Bayes posterior concentration in sparse high-dimensional linear models
• Variable selection consistency of Gaussian process regression
• Bayesian effect selection in structured additive distributional regression models
• A model selection approach for variable selection with censored data
• DOLDA: a regularized supervised topic model for high-dimensional multi-class regression
• On the exponentially weighted aggregate with the Laplace prior
• Ultra high-dimensional multivariate posterior contraction rate under shrinkage priors
• Bayesian Approaches to Shrinkage and Sparse Estimation
• Empirical priors and coverage of posterior credible sets in a sparse normal mean model
• Posterior consistency in linear models under shrinkage priors
• Probabilistic Community Detection With Unknown Number of Communities
• Rates of contraction of posterior distributions based on \(p\)-exponential priors
• Variable selection via penalized credible regions with Dirichlet-Laplace global-local shrinkage priors
• Some priors for sparse regression modelling
• High-dimensional posterior consistency for hierarchical non-local priors in regression
• High-dimensional multivariate posterior consistency under global-local shrinkage priors
• Prediction risk for the horseshoe regression
• Bayesian inference and optimal design for the sparse linear model
• Hierarchical species sampling models
• A new flexible Bayesian hypothesis test for multivariate data
• Fast Markov Chain Monte Carlo for High-Dimensional Bayesian Regression Models With Shrinkage Priors
• Comment: ``Bayes, oracle Bayes and empirical Bayes
• Posterior asymptotic normality for an individual coordinate in high-dimensional linear regression
• New approach to Bayesian high-dimensional linear regression
• Bayesian approaches to variable selection: a comparative study from practical perspectives
• A Hierarchical Expected Improvement Method for Bayesian Optimization
• A Mass-Shifting Phenomenon of Truncated Multivariate Normal Priors
• Joint Bayesian Variable and DAG Selection Consistency for High-dimensional Regression Models with Network-structured Covariates
• Bayesian fused lasso modeling via horseshoe prior
• Nearly optimal Bayesian shrinkage for high-dimensional regression
• Subjective Bayesian testing using calibrated prior probabilities
• Improved inference for doubly robust estimators of heterogeneous treatment effects
• Graph signal denoising using \(t\)-shrinkage priors
• Posterior contraction in group sparse logit models for categorical responses
• Bayesian neural networks for selection of drug sensitive genes
• Posterior contraction rates for stochastic block models
• On the sparse Bayesian learning of linear models
• Contraction properties of shrinkage priors in logistic regression
• Bayesian high-dimensional semi-parametric inference beyond sub-Gaussian errors
• Empirical Bayes inference in sparse high-dimensional generalized linear models
• Model averaging prediction by \(K\)-fold cross-validation
• Discussion to: Bayesian graphical models for modern biological applications by Y. Ni, V. Baladandayuthapani, M. Vannucci and F.C. Stingo
• Sparse Bayesian linear regression using generalized normal priors
• Two-way sparsity for time-varying networks with applications in genomics
• Bayesian Regression Using a Prior on the Model Fit: The R2-D2 Shrinkage Prior
• Bayesian inference in high-dimensional linear models using an empirical correlation-adaptive prior
• Objective Bayesian edge screening and structure selection for Ising networks
• Model Uncertainty Quantification in Cox Regression
• Random weighting in LASSO regression
• Contraction of a quasi-Bayesian model with shrinkage priors in precision matrix estimation
• Posterior impropriety of some sparse Bayesian learning models
• Adaptive variational Bayes: optimality, computation and applications
• Consistent skinny Gibbs in probit regression

Uses Software

• SSS
• EBayesThresh
• ESS++
• EMVS

This page was built for publication: Bayesian linear regression with sparse priors

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q888501)