Fast exact Bayesian inference for sparse signals in the normal sequence model

DOI10.1214/20-BA1227arXiv1810.10883OpenAlexW3046972439MaRDI QIDQ6201433FDOQ6201433

Authors: Tim van Erven, Botond Szabó 

Publication date: 20 February 2024

Published in: Bayesian Analysis (Search for Journal in Brave)

Abstract: We consider exact algorithms for Bayesian inference with model selection priors (including spike-and-slab priors) in the sparse normal sequence model. Because the best existing exact algorithm becomes numerically unstable for sample sizes over n=500, there has been much attention for alternative approaches like approximate algorithms (Gibbs sampling, variational Bayes, etc.), shrinkage priors (e.g. the Horseshoe prior and the Spike-and-Slab LASSO) or empirical Bayesian methods. However, by introducing algorithmic ideas from online sequential prediction, we show that exact calculations are feasible for much larger sample sizes: for general model selection priors we reach n=25000, and for certain spike-and-slab priors we can easily reach n=100000. We further prove a de Finetti-like result for finite sample sizes that characterizes exactly which model selection priors can be expressed as spike-and-slab priors. The computational speed and numerical accuracy of the proposed methods are demonstrated in experiments on simulated data, on a differential gene expression data set, and to compare the effect of multiple hyper-parameter settings in the beta-binomial prior. In our experimental evaluation we compute guaranteed bounds on the numerical accuracy of all new algorithms, which shows that the proposed methods are numerically reliable whereas an alternative based on long division is not.

Full work available at URL: https://arxiv.org/abs/1810.10883

zbMATH Keywords

model selectionhigh-dimensional statisticsspike-and-slab prior

Mathematics Subject Classification ID

Statistics (62-XX) Bayesian inference (62F15) Nonparametric estimation (62G05)

Cites Work

• EMVS: The EM Approach to Bayesian Variable Selection
• The horseshoe estimator for sparse signals
• Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
• Adapting to unknown sparsity by controlling the false discovery rate
• Mixtures of g Priors for Bayesian Variable Selection
• Bayesian Variable Selection in Linear Regression
• Title not available (Why is that?)
• Gaussian model selection
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• An exploration of aspects of Bayesian multiple testing
• Bayesian estimation of sparse signals with a continuous spike-and-slab prior
• Scalable variational inference for Bayesian variable selection in regression, and its accuracy in genetic association studies
• Title not available (Why is that?)
• Benchmark priors for Bayesian model averaging.
• Finite exchangeable sequences
• Title not available (Why is that?)
• The horseshoe estimator: posterior concentration around nearly black vectors
• THE TRANSFORMATION OF POISSON, BINOMIAL AND NEGATIVE-BINOMIAL DATA
• Criteria for Bayesian model choice with application to variable selection
• Fast multiplication of large numbers
• Accuracy and Stability of Numerical Algorithms
• Needles and straw in a haystack: posterior concentration for possibly sparse sequences
• Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
• Bayesian linear regression with sparse priors
• De Finetti's theorem for abstract finite exchangeable sequences
• Transformations Related to the Angular and the Square Root
• The Moment Problem
• Reconstruction of sparse vectors in white Gaussian noise
• Uncertainty quantification for the horseshoe (with discussion)
• Adaptive posterior contraction rates for the horseshoe
• Empirical Bayes analysis of spike and slab posterior distributions
• Spike and slab empirical Bayes sparse credible sets
• Needles and straw in a haystack: robust confidence for possibly sparse sequences
• Proximal Markov chain Monte Carlo algorithms
• Universal Codes From Switching Strategies
• OUP accepted manuscript

Cited In (1)

• On minimax optimality of sparse Bayes predictive density estimates

This page was built for publication: Fast exact Bayesian inference for sparse signals in the normal sequence model

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6201433)