Sketching for principal component regression
From MaRDI portal
(Redirected from Publication:5380270)
compressed least squaresleast squareslinear regressionprincipal component regressionrandomized numerical linear algebrasketching
Factor analysis and principal components; correspondence analysis (62H25) Linear regression; mixed models (62J05) Direct numerical methods for linear systems and matrix inversion (65F05) Numerical solutions to overdetermined systems, pseudoinverses (65F20) Ill-posedness and regularization problems in numerical linear algebra (65F22) Randomized algorithms (68W20)
Abstract: Principal component regression (PCR) is a useful method for regularizing linear regression. Although conceptually simple, straightforward implementations of PCR have high computational costs and so are inappropriate when learning with large scale data. In this paper, we propose efficient algorithms for computing approximate PCR solutions that are, on one hand, high quality approximations to the true PCR solutions (when viewed as minimizer of a constrained optimization problem), and on the other hand entertain rigorous risk bounds (when viewed as statistical estimators). In particular, we propose an input sparsity time algorithms for approximate PCR. We also consider computing an approximate PCR in the streaming model, and kernel PCR. Empirical results demonstrate the excellent performance of our proposed methods.
Recommendations
- On principal components regression, random projections, and column subsampling
- Principal component projection with low-degree polynomials
- A randomized algorithm for principal component analysis
- Sparse principal component regression with adaptive loading
- The principal problem with principal components regression
Cites work
- A fast randomized algorithm for overdetermined linear least-squares regression
- Blendenpik: Supercharging LAPACK's Least-Squares Solver
- Compressed matrix multiplication
- Dimensionality reduction for k-means clustering and low rank approximation
- Faster kernel ridge regression using sketching and preconditioning
- Faster least squares approximation
- Finding frequent items in data streams
- Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions
- scientific article; zbMATH DE number 47926 (Why is no real title available?)
- Iterative Hessian sketch: fast and accurate solution approximation for constrained least-squares
- LSRN: A parallel iterative solver for strongly over- or underdetermined systems
- Matrix algorithms. Vol. 2: Eigensystems
- Numerical linear algebra in the streaming model
- On principal components and regression: a statistical explanation of a natural phenomenon
- On principal components regression, random projections, and column subsampling
- On the Perturbation of Pseudo-Inverses, Projections and Linear Least Squares Problems
- Optimal Approximate Matrix Product in Terms of Stable Rank
- PRIMME: preconditioned iterative multimethod eigensolver -- methods and software description
- PRIMME\_SVDS: a high-performance preconditioned SVD solver for accurate large-scale computations
- Random Projections for Large-Scale Regression
- Randomized Sketches of Convex Programs With Sharp Guarantees
- Schubert varieties and distances between subspaces of different dimensions
- Sketched ridge regression: optimization perspective, statistical perspective, and model averaging
- Sketching as a tool for numerical linear algebra
- Structural Convergence Results for Approximation of Dominant Subspaces from Block Krylov Spaces
- The Rotation of Eigenvectors by a Perturbation. III
Cited in
(6)- Robust high-dimensional factor models with applications to statistical machine learning
- Functional principal subspace sampling for large scale functional data analysis
- Principal component projection with low-degree polynomials
- scientific article; zbMATH DE number 7307477 (Why is no real title available?)
- On Robustness of Principal Component Regression
- Sketched approximation of regularized canonical correlation analysis
This page was built for publication: Sketching for principal component regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5380270)
