Graphics processing units and high-dimensional optimization

DOI10.1214/10-STS336zbMATH Open1329.62028arXiv1003.3272OpenAlexW3101511497WikidataQ42056643 ScholiaQ42056643MaRDI QIDQ906530FDOQ906530

Authors: Hua Zhou, Marc A. Suchard, Kenneth Lange 

Publication date: 22 January 2016

Published in: Statistical Science (Search for Journal in Brave)

Abstract: This paper discusses the potential of graphics processing units (GPUs) in high-dimensional optimization problems. A single GPU card with hundreds of arithmetic cores can be inserted in a personal computer and dramatically accelerates many statistical algorithms. To exploit these devices fully, optimization algorithms should reduce to multiple parallel tasks, each accessing a limited amount of data. These criteria favor EM and MM algorithms that separate parameters and data. To a lesser extent block relaxation and coordinate descent and ascent also qualify. We demonstrate the utility of GPUs in nonnegative matrix factorization, PET image reconstruction, and multidimensional scaling. Speedups of 100 fold can easily be attained. Over the next decade, GPUs will fundamentally alter the landscape of computational statistics. It is time for more statisticians to get on-board.

Full work available at URL: https://arxiv.org/abs/1003.3272

Recommendations

• Megapixel topology optimization on a graphics processing unit
• Parallel global optimization on GPU
• GPU implementation of a multiobjective search algorithm
• Computing of high breakdown regression estimators without sorting on graphics processing units
• GPU accelerated greedy algorithms for compressed sensing

zbMATH Keywords

multidimensional scalingnonnegative matrix factorizationparallel computingblock relaxationEM and MM algorithmsPET scanning

Mathematics Subject Classification ID

Software, source code, etc. for problems pertaining to statistics (62-04) Parallel numerical computation (65Y05) Computer graphics; computational geometry (digital and algorithmic aspects) (68U05) Parallel algorithms in computer science (68W10) Numerical algorithms for specific classes of architectures (65Y10)

Cites Work

• Title not available (Why is that?)
• Maximum likelihood estimation via the ECM algorithm: A general framework
• The ECME algorithm: A simple extension of EM and ECM with faster monotone convergence
• The EM Algorithm and Extensions, 2E
• Simple and Globally Convergent Methods for Accelerating the Convergence of Any EM Algorithm
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Algorithms and applications for approximate nonnegative matrix factorization
• Title not available (Why is that?)
• Conjugate Gradient Acceleration of the EM Algorithm
• Title not available (Why is that?)
• A quasi-Newton acceleration for high-dimensional optimization algorithms
• Learning the parts of objects by non-negative matrix factorization
• Composition Markov chains of multinomial type
• The MM alternative to EM
• A Statistical Model for Positron Emission Tomography
• The tunneling method for global optimization in multidimensional scaling
• Parallel multivariate slice sampling
• Squared polynomial extrapolation methods with cycling: an application to the positron emission tomography problem
• Horseshoes in multidimensional scaling and local kernel methods

Cited In (22)

• Massive Parallelization of Massive Sample-Size Survival Analysis
• Scalable Bayesian inference for self-excitatory stochastic processes applied to big American gunfire data
• Title not available (Why is that?)
• GPU implementation of a multiobjective search algorithm
• Megapixel topology optimization on a graphics processing unit
• GPU accelerated MCMC for modeling terrorist activity
• Computing of high breakdown regression estimators without sorting on graphics processing units
• Massive parallelization of serial inference algorithms for a complex generalized linear model
• Multilevel quantile function modeling with application to birth outcomes
• Title not available (Why is that?)
• HIGH PRECISION INTEGER ADDITION, SUBTRACTION AND MULTIPLICATION WITH A GRAPHICS PROCESSING UNIT
• Variational algorithms for biclustering models
• Massive Parallelization Boosts Big Bayesian Multidimensional Scaling
• GPU parameter tuning for tall and skinny dense linear least squares problems
• Estimation for general birth-death processes
• An Asynchronous Distributed Expectation Maximization Algorithm for Massive Data: The DEM Algorithm
• Graphics card as a cheap supercomputer.
• MM algorithms for geometric and signomial programming
• Parallel statistical computing for statistical inference
• Towards a parallel component in a GPU-CUDA environment: a case study with the L-BFGS Harwell routine
• High-performance statistical computing in the computing environments of the 2020s
• High-performance processing of covariance matrices using GPU computations

Uses Software

• CUDA
• CUBLAS
• gputools

This page was built for publication: Graphics processing units and high-dimensional optimization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q906530)