A fast regression via SVD and marginalization
From MaRDI portal
Publication:2135884
Abstract: We describe a numerical scheme for evaluating the posterior moments of Bayesian linear regression models with partial pooling of the coefficients. The principal analytical tool of the evaluation is a change of basis from coefficient space to the space of singular vectors of the matrix of predictors. After this change of basis and an analytical integration, we reduce the problem of finding moments of a density over k + m dimensions, to finding moments of an m-dimensional density, where k is the number of coefficients and k + m is the dimension of the posterior. Moments can then be computed using, for example, MCMC, the trapezoid rule, or adaptive Gaussian quadrature. An evaluation of the SVD of the matrix of predictors is the dominant computational cost and is performed once during the precomputation stage. We demonstrate numerical results of the algorithm. The scheme described in this paper generalizes naturally to multilevel and multi-group hierarchical regression models where normal-normal parameters appear.
Recommendations
- ``Regression anytime with brute-force SVD truncation
- Fast regression with an \(\ell_{\infty}\) guarantee
- An improved scheme for fast approximation to least squares support vector regression
- Fast approximate L_ minimization: speeding up robust regression
- Fast learning rates for regularized regression algorithms
- An even faster algorithm for ridge regression of reduced rank data
- A faster algorithm for ridge regression of reduced rank data
- Sparse regression: scalable algorithms and empirical performance
Cites work
- Approximation theory and approximation practice
- Bayesian large-scale multiple regression with summary statistics from genome-wide association studies
- Calculating the Singular Values and Pseudo-Inverse of a Matrix
- Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions
- scientific article; zbMATH DE number 3658896 (Why is no real title available?)
- scientific article; zbMATH DE number 3390151 (Why is no real title available?)
- Matrix completion and low-rank SVD via fast alternating least squares
Cited in
(4)- Efficient reduced-rank methods for Gaussian processes with eigenfunction expansions
- Quickly Finding the Best Linear Model in High Dimensions via Projected Gradient Descent
- Fast methods for posterior inference of two-group normal-normal models
- Efficient Fourier representations of families of Gaussian processes
This page was built for publication: A fast regression via SVD and marginalization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2135884)
