Abstract: As an alternative to variable selection or shrinkage in high dimensional regression, we propose to randomly compress the predictors prior to analysis. This dramatically reduces storage and computational bottlenecks, performing well when the predictors can be projected to a low dimensional linear subspace with minimal loss of information about the response. As opposed to existing Bayesian dimensionality reduction approaches, the exact posterior distribution conditional on the compressed data is available analytically, speeding up computation by many orders of magnitude while also bypassing robustness issues due to convergence and mixing problems with MCMC. Model averaging is used to reduce sensitivity to the random projection matrix, while accommodating uncertainty in the subspace dimension. Strong theoretical support is provided for the approach by showing near parametric convergence rates for the predictive density in the large p small n asymptotic paradigm. Practical performance relative to competitors is illustrated in simulations and real data applications.
Recommendations
- Targeted random projection for prediction from high-dimensional features
- Compressed Gaussian process for manifold regression
- Random projections for Bayesian regression
- Bayesian sparse multiple regression for simultaneous rank reduction and variable selection
- Bayesian regression based on principal components for high-dimensional data
Cited in
(25)- Toward Bayesian Data Compression
- A Powerful Bayesian Test for Equality of Means in High Dimensions
- Convex non-parametric least squares, causal structures and productivity
- Detecting structural changes in longitudinal network data
- Data science, big data and statistics
- New approach to Bayesian high-dimensional linear regression
- Bayesian regression based on principal components for high-dimensional data
- Bayesian compressed vector autoregressions
- Goodness-of-fit in production models: a Bayesian perspective
- On \(b\)-bit min-wise hashing for large-scale regression and classification with sparse data
- Nowcasting GDP using machine learning methods
- Compressing parameters in Bayesian high-order models with application to logistic sequence models
- Forecasting using random subspace methods
- Compressed Gaussian process for manifold regression
- Ensemble Subset Regression (ENSURE): Efficient High-dimensional Prediction
- Calibrating covariate informed product partition models
- Comparing unconstrained parametrization methods for return covariance matrix prediction
- Bayesian data sketching for varying coefficient regression models
- Targeted random projection for prediction from high-dimensional features
- A Bayesian graphical approach for large-scale portfolio management with fewer historical data
- Influential Observations Detection by Random Projection in High-Dimensional Multivariate Response Linear Model
- Compressed covariance estimation with automated dimension learning
- Random projections for Bayesian regression
- Bayesian shrinkage in mixture-of-experts models: identifying robust determinants of class membership
- Empirical priors for prediction in sparse high-dimensional linear regression
This page was built for publication: Bayesian compressed regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5367463)
