Fast exact Bayesian inference for sparse signals in the normal sequence model

DOI10.1214/20-BA1227OpenAlexW3046972439MaRDI QIDQ6201433FDOQ6201433

Authors: Tim van Erven, Botond Szabó 

Publication date: 20 February 2024

Published in: Bayesian Analysis (Search for Journal in Brave)

Abstract: We consider exact algorithms for Bayesian inference with model selection priors (including spike-and-slab priors) in the sparse normal sequence model. Because the best existing exact algorithm becomes numerically unstable for sample sizes over n=500, there has been much attention for alternative approaches like approximate algorithms (Gibbs sampling, variational Bayes, etc.), shrinkage priors (e.g. the Horseshoe prior and the Spike-and-Slab LASSO) or empirical Bayesian methods. However, by introducing algorithmic ideas from online sequential prediction, we show that exact calculations are feasible for much larger sample sizes: for general model selection priors we reach n=25000, and for certain spike-and-slab priors we can easily reach n=100000. We further prove a de Finetti-like result for finite sample sizes that characterizes exactly which model selection priors can be expressed as spike-and-slab priors. The computational speed and numerical accuracy of the proposed methods are demonstrated in experiments on simulated data, on a differential gene expression data set, and to compare the effect of multiple hyper-parameter settings in the beta-binomial prior. In our experimental evaluation we compute guaranteed bounds on the numerical accuracy of all new algorithms, which shows that the proposed methods are numerically reliable whereas an alternative based on long division is not.

Full work available at URL: https://arxiv.org/abs/1810.10883

zbMATH Keywords

model selectionhigh-dimensional statisticsspike-and-slab prior

Mathematics Subject Classification ID

Statistics (62-XX) Bayesian inference (62F15) Nonparametric estimation (62G05)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Accuracy and Stability of Numerical Algorithms
• Adapting to unknown sparsity by controlling the false discovery rate
• Adaptive posterior contraction rates for the horseshoe
• An exploration of aspects of Bayesian multiple testing
• Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• Bayesian Variable Selection in Linear Regression
• Bayesian block-diagonal variable selection and model averaging
• Bayesian estimation of sparse signals with a continuous spike-and-slab prior
• Bayesian linear regression with sparse priors
• Benchmark priors for Bayesian model averaging.
• Criteria for Bayesian model choice with application to variable selection
• De Finetti's theorem for abstract finite exchangeable sequences
• EMVS: the EM approach to Bayesian variable selection
• Empirical Bayes analysis of spike and slab posterior distributions
• Fast multiplication of large numbers
• Finite exchangeable sequences
• Gaussian model selection
• Mixtures of g Priors for Bayesian Variable Selection
• Needles and straw in a haystack: posterior concentration for possibly sparse sequences
• Needles and straw in a haystack: robust confidence for possibly sparse sequences
• Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
• Proximal Markov chain Monte Carlo algorithms
• Reconstruction of sparse vectors in white Gaussian noise
• Scalable variational inference for Bayesian variable selection in regression, and its accuracy in genetic association studies
• Spike and slab empirical Bayes sparse credible sets
• THE TRANSFORMATION OF POISSON, BINOMIAL AND NEGATIVE-BINOMIAL DATA
• The horseshoe estimator for sparse signals
• The horseshoe estimator: posterior concentration around nearly black vectors
• The moment problem
• Transformations Related to the Angular and the Square Root
• Uncertainty quantification for the horseshoe (with discussion)
• Universal Codes From Switching Strategies

Cited In (1)

• On minimax optimality of sparse Bayes predictive density estimates

This page was built for publication: Fast exact Bayesian inference for sparse signals in the normal sequence model

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6201433)