Bayesian Compressed Regression

DOI10.1080/01621459.2014.969425zbMATH Open1373.62100arXiv1303.0642OpenAlexW1999351024MaRDI QIDQ5367463FDOQ5367463

Rajarshi Guhaniyogi, David Dunson

Publication date: 13 October 2017

Published in: Journal of the American Statistical Association (Search for Journal in Brave)

Abstract: As an alternative to variable selection or shrinkage in high dimensional regression, we propose to randomly compress the predictors prior to analysis. This dramatically reduces storage and computational bottlenecks, performing well when the predictors can be projected to a low dimensional linear subspace with minimal loss of information about the response. As opposed to existing Bayesian dimensionality reduction approaches, the exact posterior distribution conditional on the compressed data is available analytically, speeding up computation by many orders of magnitude while also bypassing robustness issues due to convergence and mixing problems with MCMC. Model averaging is used to reduce sensitivity to the random projection matrix, while accommodating uncertainty in the subspace dimension. Strong theoretical support is provided for the approach by showing near parametric convergence rates for the predictive density in the large p small n asymptotic paradigm. Practical performance relative to competitors is illustrated in simulations and real data applications.

Full work available at URL: https://arxiv.org/abs/1303.0642

zbMATH Keywords

dimensionality reductionsparsitysufficient dimension reductionlarge \(p\), small \(n\)compressed sensingdata compressionrandom projectionBayesian compressed regression

Mathematics Subject Classification ID

Bayesian inference (62F15) General nonlinear regression (62J02) Ridge regression; shrinkage estimators (Lasso) (62J07)

Cited In (15)

• Convex non-parametric least squares, causal structures and productivity
• Detecting structural changes in longitudinal network data
• Data science, big data and statistics
• Bayesian compressed vector autoregressions
• Targeted Random Projection for Prediction From High-Dimensional Features
• Forecasting using random subspace methods
• Ensemble Subset Regression (ENSURE): Efficient High-dimensional Prediction
• Title not available (Why is that?)
• Calibrating covariate informed product partition models
• Comparing unconstrained parametrization methods for return covariance matrix prediction
• A Bayesian graphical approach for large-scale portfolio management with fewer historical data
• Compressed covariance estimation with automated dimension learning
• Random projections for Bayesian regression
• Bayesian shrinkage in mixture-of-experts models: identifying robust determinants of class membership
• A Powerful Bayesian Test for Equality of Means in High Dimensions

This page was built for publication: Bayesian Compressed Regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5367463)