Sparse principal component regression for generalized linear models

DOI10.1016/J.CSDA.2018.03.008zbMATH Open1469.62091arXiv1609.08886OpenAlexW2526315116WikidataQ130083174 ScholiaQ130083174MaRDI QIDQ1662867FDOQ1662867

Toyoyuki Takada, Hironori Fujisawa, Shuichi Kawano, Toshihiko Shiroishi

Publication date: 20 August 2018

Published in: Computational Statistics and Data Analysis (Search for Journal in Brave)

Abstract: Principal component regression (PCR) is a widely used two-stage procedure: principal component analysis (PCA), followed by regression in which the selected principal components are regarded as new explanatory variables in the model. Note that PCA is based only on the explanatory variables, so the principal components are not selected using the information on the response variable. In this paper, we propose a one-stage procedure for PCR in the framework of generalized linear models. The basic loss function is based on a combination of the regression loss and PCA loss. An estimate of the regression parameter is obtained as the minimizer of the basic loss function with a sparse penalty. We call the proposed method sparse principal component regression for generalized linear models (SPCR-glm). Taking the two loss function into consideration simultaneously, SPCR-glm enables us to obtain sparse principal component loadings that are related to a response variable. However, a combination of loss functions may cause a parameter identification problem, but this potential problem is avoided by virtue of the sparse penalty. Thus, the sparse penalty plays two roles in this method. The parameter estimation procedure is proposed using various update algorithms with the coordinate descent algorithm. We apply SPCR-glm to two real datasets, doctor visits data and mouse consomic strain data. SPCR-glm provides more easily interpretable principal component (PC) scores and clearer classification on PC plots than the usual PCA.

Full work available at URL: https://arxiv.org/abs/1609.08886

Recommendations

• Sparse principal component regression with adaptive loading
• Performances of some high dimensional regression methods: sparse principal component regression
• Bayesian principal component regression with data-driven component selection
• Principal component estimation for generalized linear regression
• Principal component-guided sparse regression

zbMATH Keywords

variable selectiondimension reductioncoordinate descentsparse regularization

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Factor analysis and principal components; correspondence analysis (62H25) Applications of statistics to biology and medical sciences; meta analysis (62P10) Generalized linear models (logistic models) (62J12)

Cites Work

• A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis
• The Adaptive Lasso and Its Oracle Properties
• Principal component analysis.
• Least angle regression. (With discussion)
• Pathwise coordinate optimization
• Coordinate descent algorithms for lasso penalized regression
• Title not available (Why is that?)
• Title not available (Why is that?)
• Regularization and Variable Selection Via the Elastic Net
• Functional Principal Component Regression and Functional Partial Least Squares
• A Direct Formulation for Sparse PCA Using Semidefinite Programming
• Rotation to simple loadings using component loss functions: the oblique case
• A Statistical View of Some Chemometrics Regression Tools
• PLS generalised linear regression
• Classification of gene microarrays by penalized logistic regression
• Sparse estimation via nonconcave penalized likelihood in factor analysis model
• A penalized maximum likelihood approach to sparse factor analysis
• Kernel PCA for feature extraction and de-noising in nonlinear regression
• Sparse principal component regression with adaptive loading

Cited In (9)

• Hierarchical clustered multiclass discriminant analysis via cross-validation
• Fast deflation sparse principal component analysis via subspace projections
• Sparse principal component regression via singular value decomposition approach
• Principal component estimation for generalized linear regression
• Sparse principal component regression with adaptive loading
• Estimation and visualization of heterogeneous treatment effects for multiple outcomes
• Sparse Estimation of Generalized Linear Models (GLM) via Approximated Information Criteria
• Penalized principal logistic regression for sparse sufficient dimension reduction
• Title not available (Why is that?)

Uses Software

• R
• glmnet
• ElemStatLearn
• PMA

This page was built for publication: Sparse principal component regression for generalized linear models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1662867)