Random projections for Bayesian regression

DOI10.1007/S11222-015-9608-ZzbMATH Open1505.62157arXiv1504.06122OpenAlexW2152435742WikidataQ59602664 ScholiaQ59602664MaRDI QIDQ144017FDOQ144017

Authors: Leo N. Geppert, Katja Ickstadt, Alexander Munteanu, Jens Quedenfeld, Christian Sohler, Leo N. Geppert, Alexander Munteanu, Jens Quedenfeld, Katja Ickstadt, Christian Sohler 

Publication date: 19 November 2015

Published in: Statistics and Computing (Search for Journal in Brave)

Abstract: This article deals with random projections applied as a data reduction technique for Bayesian regression analysis. We show sufficient conditions under which the entire $d$-dimensional distribution is approximately preserved under random projections by reducing the number of data points from $n$ to $kinO(operatornamepoly(d/varepsilon))$ in the case $nggd$. Under mild assumptions, we prove that evaluating a Gaussian likelihood function based on the projected data instead of the original data yields a $(1+O(varepsilon))$-approximation in terms of the $el{l}_2$ Wasserstein distance. Our main result shows that the posterior distribution of Bayesian linear regression is approximated up to a small error depending on only an $varepsilon$-fraction of its defining parameters. This holds when using arbitrary Gaussian priors or the degenerate case of uniform distributions over $mathbb{R}^d$ for $$. Our empirical evaluations involve different simulated settings of Bayesian linear regression. Our experiments underline that the proposed method is able to recover the regression model up to small error while considerably reducing the total running time.

Full work available at URL: https://arxiv.org/abs/1504.06122

Recommendations

• Bayesian inference via projections
• Random Projections for Large-Scale Regression
• The regression estimation with random projection
• scientific article; zbMATH DE number 6276198
• Bayesian Inference with Projected Densities
• Bayesian estimation for nonparametric regression
• scientific article; zbMATH DE number 3907571
• Nonparametric Bayesian regression
• Bayesian shrinkage prediction for the regression problem
• scientific article; zbMATH DE number 2143289

zbMATH Keywords

random projectionsBayesian regressiondata reductionposterior approximation

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Bayesian inference (62F15) Linear regression; mixed models (62J05) Randomized algorithms (68W20)

Cites Work

• Bayesian computing with INLA: new features
• The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo
• Principal component analysis.
• Bayesian data analysis.
• Title not available (Why is that?)
• Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations (with discussion)
• Bayesian Compressed Regression
• Pattern recognition and machine learning.
• Improved matrix algorithms via the subsampled randomized Hadamard transform
• Title not available (Why is that?)
• Optimal Transport
• Prior distributions for variance parameters in hierarchical models (Comment on article by Browne and Draper)
• Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions
• A simple proof of the restricted isometry property for random matrices
• Fast dimension reduction using Rademacher series on dual BCH codes
• A class of Wasserstein metrics for probability distributions
• The space complexity of approximating the frequency moments
• Random projections for Bayesian regression
• Likelihood-based data squashing: A modeling approach to instance construction
• Numerical methods for solving linear least squares problems
• The fast Cauchy transform and faster robust linear regression
• Communication-optimal parallel and sequential QR and LU factorizations
• Dimensionality Reduction for k-Means Clustering and Low Rank Approximation
• Low-Rank Approximation and Regression in Input Sparsity Time
• Data Streams: Algorithms and Applications
• Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information
• Sampling algorithms for \(l_2\) regression and applications
• Sampling Algorithms and Coresets for $\ell_p$ Regression
• Spectral algorithms
• A Reliable Randomized Algorithm for the Closest-Pair Problem
• Title not available (Why is that?)
• Twice-Ramanujan sparsifiers
• Lower Bounds for Oblivious Subspace Embeddings
• Numerical linear algebra in the streaming model
• Speeding Up MCMC by Efficient Data Subsampling
• The Johnson-Lindenstrauss Transform: An Empirical Study
• Near-Optimal Coresets for Least-Squares Regression
• Sketching for M-Estimators: A Unified Approach to Robust Regression
• Efficient Gaussian process regression for large datasets
• Sparsity lower bounds for dimensionality reducing maps
• Title not available (Why is that?)
• Turning Big data into tiny data: Constant-size coresets for k-means, PCA and projective clustering
• Compressed sensing
• A one-pass sequential Monte Carlo method for Bayesian analysis of massive datasets
• Faster least squares approximation

Cited In (8)

• Improving Random Projections Using Marginal Information
• Title not available (Why is that?)
• On randomized sketching algorithms and the Tracy-Widom law
• Optimal projection of observations in a Bayesian setting
• Beta Random Projection
• Title not available (Why is that?)
• RaProR
• Random projections for Bayesian regression

Uses Software

• R
• UCI-ml
• R-INLA
• Stan
• BayesDA
• ElemStatLearn
• abc
• RaProR

This page was built for publication: Random projections for Bayesian regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q144017)