Distributed Bayesian matrix factorization with limited communication

DOI10.1007/S10994-019-05778-2MaRDI QIDQ2320545zbMATH OpenOpenAlexFDO

Authors Xiangju Qin, Paul Blomstedt, Eemeli Leppäaho, Pekka Parviainen, Samuel Kaski

Publication date 23 August 2019

Published in Machine Learning (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://arxiv.org/abs/1703.00734

zbMATH Keywords

distributed inferenceBayesian matrix factorizationembarrassingly parallel MCMCposterior propagation

Mathematics Subject Classification ID

Bayesian inference (62F15) Factor analysis and principal components; correspondence analysis (62H25) Learning and adaptive systems in artificial intelligence (68T05) Factorization of matrices (15A23)

Abstract: Bayesian matrix factorization (BMF) is a powerful tool for producing low-rank representations of matrices and for predicting missing values and providing confidence intervals. Scaling up the posterior inference for massive-scale matrices is challenging and requires distributing both data and computation over many workers, making communication the main computational bottleneck. Embarrassingly parallel inference would remove the communication needed, by using completely independent computations on different data subsets, but it suffers from the inherent unidentifiability of BMF solutions. We introduce a hierarchical decomposition of the joint posterior distribution, which couples the subset inferences, allowing for embarrassingly parallel computations in a sequence of at most three stages. Using an efficient approximate implementation, we show improvements empirically on both real and simulated data. Our distributed approach is able to achieve a speed-up of almost an order of magnitude over the full posterior, with a negligible effect on predictive accuracy. Our method outperforms state-of-the-art embarrassingly parallel MCMC methods in accuracy, and achieves results competitive to other available distributed and parallel implementations of BMF.

Recommendations

• Distributed matrix completion and robust factorization
• scientific article; zbMATH DE number 7164690
• Bayesian hierarchical kernelized probabilistic matrix factorization
• Theoretical analysis of Bayesian matrix factorization
• Scaling up Bayesian variational inference using distributed computing clusters

Cites work

• scientific article; zbMATH DE number 2063756 (Why is no real title available?)
• Expectation propagation as a way of life: a framework for Bayesian inference on partitioned data
• Merging MCMC subposteriors through Gaussian-process approximations
• Patterns of scalable Bayesian inference
• Robust and scalable Bayes via a median of subset posterior measures
• Scalable Bayes via barycenter in Wasserstein space
• Sparse Bayesian infinite factor models

Cited in

(3)

• Distributed Bayesian machine learning procedures
• Distributed matrix completion and robust factorization
• Bayesian hierarchical kernelized probabilistic matrix factorization

Describes a project that uses

Uses Software

• MovieLens
• ChEMBL

This page was built for publication: Distributed Bayesian matrix factorization with limited communication

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2320545)