GPU-accelerated Gibbs sampling: a case study of the horseshoe probit model

DOI10.1007/S11222-018-9809-3MaRDI QIDQ2329768zbMATH OpenOpenAlexWikidataFDO

Authors Alexander Terenin, Shawfeng Dong, David Draper

Publication date 18 October 2019

Published in Statistics and Computing (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://arxiv.org/abs/1608.04329

zbMATH Keywords

Bayesian generalized linear modelsbig datagraphics processing unitshigh-dimensional statistical modelingMarkov chain Monte Carloparallel computing

Mathematics Subject Classification ID

Graphical methods in statistics (62A09) Generalized linear models (logistic models) (62J12) Statistical aspects of big data and data science (62R07) Monte Carlo methods (65C05)

Abstract: Gibbs sampling is a widely used Markov chain Monte Carlo (MCMC) method for numerically approximating integrals of interest in Bayesian statistics and other mathematical sciences. Many implementations of MCMC methods do not extend easily to parallel computing environments, as their inherently sequential nature incurs a large synchronization cost. In the case study illustrated by this paper, we show how to do Gibbs sampling in a fully data-parallel manner on a graphics processing unit, for a large class of exchangeable models that admit latent variable representations. Our approach takes a systems perspective, with emphasis placed on efficient use of compute hardware. We demonstrate our method on a Horseshoe Probit regression model and find that our implementation scales effectively to thousands of predictors and millions of data points simultaneously.

Recommendations

• scientific article; zbMATH DE number 472921
• scientific article; zbMATH DE number 549940
• Parallel multivariate slice sampling
• Independent sampling for Bayesian normal conditional autoregressive models with OpenCL acceleration
• Parallel MCMC with generalized elliptical slice sampling

Cites work

• 10.1162/jmlr.2003.3.4-5.993
• A note on geometric ergodicity and floating-point roundoff error
• Bayesian Analysis of Binary and Polychotomous Response Data
• Equation of state calculations by fast computing machines
• Leave Pima Indians alone: binary regression as a benchmark for Bayesian computation
• Mersenne twister
• Monte Carlo sampling methods using Markov chains and their applications
• Sparse Partially Collapsed MCMC for Parallel Inference in Topic Models
• Statistical Analysis of Financial Data in S-Plus
• Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
• Teaching Statistics at Google-Scale
• TestU01
• The horseshoe estimator for sparse signals
• The no-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo

Cited in

(14)

• GPU accelerated MCMC for modeling terrorist activity
• Independent sampling for Bayesian normal conditional autoregressive models with OpenCL acceleration
• Comment: A brief survey of the current state of play for Bayesian computation in data science at big-data scale
• Tree ensembles with rule structured horseshoe regularization
• A simple method for rejection sampling efficiency improvement on SIMT architectures
• Contraction of a quasi-Bayesian model with shrinkage priors in precision matrix estimation
• Lasso meets horseshoe: a survey
• Fast parallel \(\alpha \)-stable distribution function evaluation and parameter estimation using OpenCL in GPGPUs
• Massive parallelization of serial inference algorithms for a complex generalized linear model
• Horseshoe Regularisation for Machine Learning in Complex and Deep Models¹
• Discussion
• Vector operations for accelerating expensive Bayesian computations - a tutorial guide
• Massive Parallelization of Massive Sample-Size Survival Analysis
• Group inverse-gamma gamma shrinkage for sparse linear models with block-correlated regressors

Describes a project that uses

Uses Software

• MASS (R)
• MersenneTwister
• Spark
• NUTS
• PyTorch
• R
• S-PLUS
• CUDA
• Vowpal Wabbit
• TestU01
• OpenCL
• TensorFlow

This page was built for publication: GPU-accelerated Gibbs sampling: a case study of the horseshoe probit model

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2329768)