Sparsity information and regularization in the horseshoe and other shrinkage priors

DOI10.1214/17-EJS1337SIzbMATH Open1459.62141arXiv1707.01694OpenAlexW2731461838WikidataQ61703734 ScholiaQ61703734MaRDI QIDQ86320FDOQ86320

Authors: Juho Piironen, Juho Piironen, Aki Vehtari 

Publication date: 1 January 2017

Published in: Electronic Journal of Statistics (Search for Journal in Brave)

Abstract: The horseshoe prior has proven to be a noteworthy alternative for sparse Bayesian estimation, but has previously suffered from two problems. First, there has been no systematic way of specifying a prior for the global shrinkage hyperparameter based on the prior information about the degree of sparsity in the parameter vector. Second, the horseshoe prior has the undesired property that there is no possibility of specifying separately information about sparsity and the amount of regularization for the largest coefficients, which can be problematic with weakly identified parameters, such as the logistic regression coefficients in the case of data separation. This paper proposes solutions to both of these problems. We introduce a concept of effective number of nonzero parameters, show an intuitive way of formulating the prior for the global hyperparameter based on the sparsity assumptions, and argue that the previous default choices are dubious based on their tendency to favor solutions with more unshrunk parameters than we typically expect a priori. Moreover, we introduce a generalization to the horseshoe prior, called the regularized horseshoe, that allows us to specify a minimum level of regularization to the largest values. We show that the new prior can be considered as the continuous counterpart of the spike-and-slab prior with a finite slab width, whereas the original horseshoe resembles the spike-and-slab with an infinitely wide slab. Numerical experiments on synthetic and real world data illustrate the benefit of both of these theoretical advances.

Full work available at URL: https://arxiv.org/abs/1707.01694

Recommendations

• Inducing sparsity via the horseshoe prior in imaging problems
• Theoretical Guarantees for the Horseshoe and Other Global-Local Shrinkage Priors
• Bayesian Approaches to Shrinkage and Sparse Estimation
• The horseshoe estimator for sparse signals
• Some priors for sparse regression modelling
• Functional Horseshoe Priors for Subspace Shrinkage
• Dirichlet-Laplace priors for optimal shrinkage
• The horseshoe+ estimator of ultra-sparse signals
• Nonparametric sparsity and regularization
• Sparsity and the possibility of inference

zbMATH Keywords

Bayesian inferencelogistic regressionsparse estimationhorseshoe priorshrinkage priors

Mathematics Subject Classification ID

Bayesian inference (62F15) Ridge regression; shrinkage estimators (Lasso) (62J07) Generalized linear models (logistic models) (62J12)

Cited In (76)

• Bayesian compositional generalized linear models for analyzing microbiome data
• Bayesian approaches to variable selection: a comparative study from practical perspectives
• Variable selection for high-dimensional incomplete data using horseshoe estimation with data augmentation
• Efficient estimation and correction of selection-induced bias with order statistics
• Efficient Posterior Sampling for Bayesian Poisson Regression
• Empirical Bayes inference in sparse high-dimensional generalized linear models
• Bayesian regularization: from Tikhonov to horseshoe
• Cauchy and other shrinkage priors for logistic regression in the presence of separation
• Ultimate Pólya Gamma Samplers–Efficient MCMC for Possibly Imbalanced Binary and Categorical Data
• Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy
• Scalable multiple network inference with the joint graphical horseshoe
• The Bayesian simulation study (BASIS) framework for simulation studies in statistical and methodological research
• The reciprocal Bayesian Lasso
• Gender inequalities at work in Southern Europe
• Modeling basal body temperature data using horseshoe process regression
• Precision matrix estimation under the horseshoe-like prior-penalty dual
• BHAFT: Bayesian heredity-constrained accelerated failure time models for detecting gene-environment interactions in survival analysis
• Global–local shrinkage multivariate logit-beta priors for multiple response-type data
• Extended beta models for poverty mapping. An application integrating survey and remote sensing data in Bangladesh
• Sparse Bayesian predictive modelling of tumour response using radiomic features
• Neuronized Priors for Bayesian Sparse Linear Regression
• The horseshoe+ estimator of ultra-sparse signals
• A horseshoe mixture model for Bayesian screening with an application to light sheet fluorescence microscopy in brain imaging
• Fast Bayesian inference on spectral analysis of multivariate stationary time series
• Generalized double Pareto shrinkage
• Variance prior forms for high-dimensional Bayesian variable selection
• The horseshoe estimator: posterior concentration around nearly black vectors
• Shrinkage priors for Bayesian penalized regression
• Mean field variational Bayes for continuous sparse signal shrinkage: pitfalls and remedies
• Empirical Bayesian Inference Using a Support Informed Prior
• Projective inference in high-dimensional problems: prediction and feature selection
• A Dynamic Binary Probit Model with Time-Varying Parameters and Shrinkage Prior
• Bayesian Lesion Estimation with a Structured Spike-and-Slab Prior
• Screening Methods for Linear Errors-in-Variables Models in High Dimensions
• Bayesian approaches to variable selection in mixture models with application to disease clustering
• Reversible Jump PDMP Samplers for Variable Selection
• Prior distributions for objective Bayesian analysis
• Bayesian deconvolution and quantification of metabolites from \(J\)-resolved NMR spectroscopy
• Bayesian sparse convex clustering via global-local shrinkage priors
• Geometric ergodicity of Gibbs samplers for the horseshoe and its regularized variants
• The role of passing network indicators in modeling football outcomes: an application using Bayesian hierarchical models
• Default Bayesian analysis with global-local shrinkage priors
• Intuitive joint priors for Bayesian linear multilevel models: the R2D2M2 prior
• Title not available (Why is that?)
• The horseshoe estimator for sparse signals
• A sparse Bayesian hierarchical vector autoregressive model for microbial dynamics in a wastewater treatment plant
• A fully Bayesian sparse polynomial chaos expansion approach with joint priors on the coefficients and global selection of terms
• Implicitly adaptive importance sampling
• Default priors for the intercept parameter in logistic regressions
• Bayesian sparse estimation using double Lomax priors
• Horseshoe shrinkage methods for Bayesian fusion estimation
• Time fused coefficient SIR model with application to COVID-19 epidemic in the United States
• Lasso meets horseshoe: a survey
• Bayesian Joint Modeling of Multiple Brain Functional Networks
• Title not available (Why is that?)
• A comparative study on high-dimensional bayesian regression with binary predictors
• Bayesian inference over the Stiefel manifold via the Givens representation
• Fully Bayesian logistic regression with hyper-LASSO priors for high-dimensional feature selection
• A spatial mixed-effects regression model for electoral data
• Horseshoe Priors for Edge-Preserving Linear Bayesian Inversion
• Functional Horseshoe Priors for Subspace Shrinkage
• Adaptive posterior contraction rates for the horseshoe
• hsstan
• nestedcv
• Marginally calibrated response distributions for end-to-end learning in autonomous driving
• Using reference models in variable selection
• Estimating the stillbirth rate for 195 countries using a Bayesian sparse regression model with temporal smoothing
• Some priors for sparse regression modelling
• A global-local approach for detecting hotspots in multiple-response regression
• Modeling obesity rate with spatial auto-correlation: a case study
• Inconsistency identification in network meta-analysis via stochastic search variable selection
• Tree ensembles with rule structured horseshoe regularization
• Informative Bayesian neural network priors for weak signals
• Shrinkage with shrunken shoulders: Gibbs sampling shrinkage model posteriors with guaranteed convergence rates
• Prediction risk for the horseshoe regression
• A Review of Data‐Driven Discovery for Dynamic Systems

This page was built for publication: Sparsity information and regularization in the horseshoe and other shrinkage priors

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q86320)