Principal component-guided sparse regression

DOI10.48550/ARXIV.1810.04651zbMATH Open1492.62116arXiv1810.04651OpenAlexW3152849944MaRDI QIDQ135079FDOQ135079

Authors: J. Kenneth Tay, Jerome Friedman, Robert Tibshirani, Jing-Yi Tay, Robert Tibshirani, Jerome H. Friedman 

Publication date: 10 October 2018

Published in: The Canadian Journal of Statistics (Search for Journal in Brave)

Abstract: We propose a new method for supervised learning, especially suited to wide data where the number of features is much greater than the number of observations. The method combines the lasso ($el{l}_1$) sparsity penalty with a quadratic penalty that shrinks the coefficient vector toward the leading principal components of the feature matrix. We call the proposed method the "principal components lasso" ("pcLasso"). The method can be especially powerful if the features are pre-assigned to groups (such as cell-pathways, assays or protein interaction networks). In that case, pcLasso shrinks each group-wise component of the solution toward the leading principal components of that group. In the process, it also carries out selection of the feature groups. We provide some theory for this method and illustrate it on a number of simulated and real data examples.

Full work available at URL: https://arxiv.org/abs/1810.04651

Recommendations

• A component Lasso
• Targeted principal components regression
• Sparse principal component regression with adaptive loading
• Sparse principal component regression for generalized linear models
• Projection sparse principal component analysis: an efficient least squares method

zbMATH Keywords

Lassoprincipal componentssupervised learningsparsityfeature group selection

Mathematics Subject Classification ID

Factor analysis and principal components; correspondence analysis (62H25) Ridge regression; shrinkage estimators (Lasso) (62J07)

Cited In (12)

• An analytical shrinkage estimator for linear regression
• Penalized orthogonal-components regression for large \(p\) small \(n\) data
• A component Lasso
• Sparse principal component regression via singular value decomposition approach
• Sparse principal component regression for generalized linear models
• Sparse principal component regression with adaptive loading
• Testing significance of features by lassoed principal components
• A guide for sparse PCA: model comparison and applications
• pcLasso
• Dualize, split, randomize: toward fast nonsmooth optimization algorithms
• A simple method to improve principal components regression
• Alternative penalty functions for penalized likelihood principal components

This page was built for publication: Principal component-guided sparse regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q135079)