Dirichlet-Laplace priors for optimal shrinkage

DOI10.1080/01621459.2014.960967zbMATH Open1373.62368arXiv1401.5398OpenAlexW2150149003WikidataQ36715879 ScholiaQ36715879MaRDI QIDQ5367461FDOQ5367461

Authors: Anirban Bhattacharya, Debdeep Pati, Natesh S. Pillai, David Dunson 

Publication date: 13 October 2017

Published in: Journal of the American Statistical Association (Search for Journal in Brave)

Abstract: Penalized regression methods, such as $L_1$ regularization, are routinely used in high-dimensional applications, and there is a rich literature on optimality properties under sparsity assumptions. In the Bayesian paradigm, sparsity is routinely induced through two-component mixture priors having a probability mass at zero, but such priors encounter daunting computational problems in high dimensions. This has motivated an amazing variety of continuous shrinkage priors, which can be expressed as global-local scale mixtures of Gaussians, facilitating computation. In sharp contrast to the frequentist literature, little is known about the properties of such priors and the convergence and concentration of the corresponding posterior distribution. In this article, we propose a new class of Dirichlet--Laplace (DL) priors, which possess optimal posterior concentration and lead to efficient posterior computation exploiting results from normalized random measure theory. Finite sample performance of Dirichlet--Laplace priors relative to alternatives is assessed in simulated and real data examples.

Full work available at URL: https://arxiv.org/abs/1401.5398

Recommendations

• Sub-optimality of some continuous shrinkage priors
• Contraction properties of shrinkage priors in logistic regression
• Bayesian Regression Using a Prior on the Model Fit: The R2-D2 Shrinkage Prior
• Generalized double Pareto shrinkage
• Variable selection via penalized credible regions with Dirichlet-Laplace global-local shrinkage priors

zbMATH Keywords

LassoBayesianregularizationhigh-dimensionalpenalized regressionconvergence rateshrinkage priorDirichlet-Laplace\(L_1\)

Mathematics Subject Classification ID

Bayesian inference (62F15) Ridge regression; shrinkage estimators (Lasso) (62J07)

Cited In (only showing first 100 items - show all)

• The Bayesian group Lasso for confounded spatial data
• Neuronized Priors for Bayesian Sparse Linear Regression
• Adaptive Bayesian nonparametric regression using a kernel mixture of polynomials with application to partial linear models
• Model selection using mass-nonlocal prior
• Radio-iBAG: radiomics-based integrative Bayesian analysis of multiplatform genomic data
• Penalising model component complexity: a principled, practical approach to constructing priors
• Spike and slab empirical Bayes sparse credible sets
• Predictive performance of Dirichlet process shrinkage methods in linear regression
• Efficient Sampling for Gaussian Linear Regression With Arbitrary Priors
• Conditions for posterior contraction in the sparse normal means problem
• Projective inference in high-dimensional problems: prediction and feature selection
• Sub-optimality of some continuous shrinkage priors
• Bayesian approaches to variable selection in mixture models with application to disease clustering
• Bayesian estimation of sparse signals with a continuous spike-and-slab prior
• Nearly optimal Bayesian shrinkage for high-dimensional regression
• Prior distributions for objective Bayesian analysis
• Title not available (Why is that?)
• Graph signal denoising using \(t\)-shrinkage priors
• Spatial Shrinkage Via the Product Independent Gaussian Process Prior
• Bayesian deconvolution and quantification of metabolites from \(J\)-resolved NMR spectroscopy
• Gradient-Based Markov Chain Monte Carlo for Bayesian Inference With Non-differentiable Priors
• Frequentist conditional variance for nonlinear mixed-effects models
• Robust sparse Bayesian infinite factor models
• Bayesian Factor Analysis for Inference on Interactions
• Fast and accurate variational inference for large Bayesian VARs with stochastic volatility
• Contraction properties of shrinkage priors in logistic regression
• A hybrid scan Gibbs sampler for Bayesian models with latent variables
• Bayesian Semiparametric Functional Mixed Models for Serially Correlated Functional Data, With Application to Glaucoma Data
• Locally adaptive smoothing with Markov random fields and shrinkage priors
• Optimal Gaussian approximations to the posterior for log-linear models with Diaconis-Ylvisaker priors
• Bayesian factor-adjusted sparse regression
• Non-parametric comparison and classification of two large-scale populations
• Bayesian wavelet shrinkage with beta priors
• Shrinkage priors for single-spiked covariance models
• Bayesian bridge regression
• MIMIX: A Bayesian Mixed-Effects Model for Microbiome Data From Designed Experiments
• The horseshoe-like regularization for feature subset selection
• Bayesian discriminant analysis using a high dimensional predictor
• Sparse linear mixed model selection via streamlined variational Bayes
• Lasso meets horseshoe: a survey
• High-dimensional confounding adjustment using continuous Spike and Slab priors
• Dynamic variable selection with spike-and-slab process priors
• Intuitive joint priors for variance parameters
• Dealing with Measurement Uncertainties as Nuisance Parameters in Bayesian Model Calibration
• Singular value shrinkage priors for Bayesian prediction
• Bayesian inference in nonparanormal graphical models
• Bayesian Regression Trees for High-Dimensional Prediction and Variable Selection
• Ultra high-dimensional multivariate posterior contraction rate under shrinkage priors
• Computationally efficient inference in large Bayesian mixed frequency VARs
• Trace Class Markov Chains for Bayesian Inference with Generalized Double Pareto Shrinkage Priors
• Bayesian Approaches to Shrinkage and Sparse Estimation
• Functional Horseshoe Priors for Subspace Shrinkage
• Sparsity information and regularization in the horseshoe and other shrinkage priors
• A novel Bayesian approach for variable selection in linear regression models
• Bayesian Semiparametric Multivariate Density Deconvolution
• Title not available (Why is that?)
• Title not available (Why is that?)
• The Spike-and-Slab LASSO
• Bayesian Bootstrap Spike-and-Slab LASSO
• Spike-and-Slab Group Lassos for Grouped Regression and Sparse Generalized Additive Models
• The illusion of the illusion of sparsity: an exercise in prior sensitivity
• Variable selection via penalized credible regions with Dirichlet-Laplace global-local shrinkage priors
• Skinny Gibbs: A Consistent and Scalable Gibbs Sampler for Model Selection
• Bayesian shrinkage in mixture-of-experts models: identifying robust determinants of class membership
• On minimax optimality of sparse Bayes predictive density estimates
• Shared Bayesian variable shrinkage in multinomial logistic regression
• Fast Monte Carlo Markov chains for Bayesian shrinkage models with random effects
• Uncertainty Quantification for Modern High-Dimensional Regression via Scalable Bayesian Methods
• The Graphical Horseshoe Estimator for Inverse Covariance Matrices
• Variable selection using shrinkage priors
• Prediction risk for the horseshoe regression
• Structured Shrinkage Priors
• Optimal false discovery control of minimax estimators
• Fast Markov Chain Monte Carlo for High-Dimensional Bayesian Regression Models With Shrinkage Priors
• Bayesian modeling of interaction between features in sparse multivariate count data with application to microbiome study
• A horseshoe mixture model for Bayesian screening with an application to light sheet fluorescence microscopy in brain imaging
• Bayesian approaches to variable selection: a comparative study from practical perspectives
• Bayesian wavelet shrinkage with logistic prior
• Asymmetric Prior in Wavelet Shrinkage
• Wavelet shrinkage in nonparametric regression models with positive noise
• Variable selection using Bayesian additive regression trees
• Transport Monte Carlo: High-Accuracy Posterior Approximation via Random Transport
• Discussions
• Revisiting Jeffreys’ Example: Bayes Test of the Normal Mean
• Bayesian high-dimensional semi-parametric inference beyond sub-Gaussian errors
• Empirical estimates for heteroscedastic hierarchical dynamic normal models
• Bayesian hierarchical modeling: application towards production results in the Eagle Ford Shale of South Texas
• Bayesian sparse convex clustering via global-local shrinkage priors
• Geometric ergodicity of Gibbs samplers for the horseshoe and its regularized variants
• Intuitive joint priors for Bayesian linear multilevel models: the R2D2M2 prior
• Bayesian Edge Regression in Undirected Graphical Models to Characterize Interpatient Heterogeneity in Cancer
• Bayesian Regression Using a Prior on the Model Fit: The R2-D2 Shrinkage Prior
• Bayesian spatial and spatiotemporal models based on multiscale factorizations
• The how and why of Bayesian nonparametric causal inference
• Targeted Random Projection for Prediction From High-Dimensional Features
• Comparison and contrast of two general functional regression modelling frameworks
• Large-scale multiple hypothesis testing with the normal-beta prime prior
• Inducing Sparsity and Shrinkage in Time-Varying Parameter Models
• A sparse factor model for clustering high-dimensional longitudinal data
• Two Metropolis--Hastings Algorithms for Posterior Measures with Non-Gaussian Priors in Infinite Dimensions

This page was built for publication: Dirichlet-Laplace priors for optimal shrinkage

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5367461)