The Degrees of Freedom of Partial Least Squares Regression

DOI10.1198/JASA.2011.TM10107zbMATH Open1232.62099arXiv1002.4112OpenAlexW2137211388WikidataQ112778551 ScholiaQ112778551MaRDI QIDQ86894FDOQ86894

Authors: Nicole Krämer, Masashi Sugiyama, Nicole Krämer, Masashi Sugiyama 

Publication date: June 2011

Published in: Journal of the American Statistical Association (Search for Journal in Brave)

Abstract: The derivation of statistical properties for Partial Least Squares regression can be a challenging task. The reason is that the construction of latent components from the predictor variables also depends on the response variable. While this typically leads to good performance and interpretable models in practice, it makes the statistical analysis more involved. In this work, we study the intrinsic complexity of Partial Least Squares Regression. Our contribution is an unbiased estimate of its Degrees of Freedom. It is defined as the trace of the first derivative of the fitted values, seen as a function of the response. We establish two equivalent representations that rely on the close connection of Partial Least Squares to matrix decompositions and Krylov subspace techniques. We show that the Degrees of Freedom depend on the collinearity of the predictor variables: The lower the collinearity is, the higher the Degrees of Freedom are. In particular, they are typically higher than the naive approach that defines the Degrees of Freedom as the number of components. Further, we illustrate how the Degrees of Freedom approach can be used for the comparison of different regression methods. In the experimental section, we show that our Degrees of Freedom estimate in combination with information criteria is useful for model selection.

Full work available at URL: https://arxiv.org/abs/1002.4112

Recommendations

• On the degrees of freedom of mixed matrix regression
• scientific article; zbMATH DE number 1058079
• On the structure of partial least squares regression
• Calculating degrees of freedom in multivariate local polynomial regression
• On the degrees of freedom in shape-restricted regression.
• Partial least squares for dependent data
• Degrees of freedom in lasso problems
• scientific article; zbMATH DE number 4174160

zbMATH Keywords

model selectiondimensionality reductioninformation criteriaKrylov subspaces

Mathematics Subject Classification ID

General nonlinear regression (62J02) Estimation in multivariate analysis (62H12)

Cited In (12)

• PLS for Big Data: a unified parallel algorithm for regularised group PLS
• Partial least squares algorithms and methods
• Near optimal prediction from relevant components
• A New Bootstrap-Based Stopping Criterion in PLS Components Construction
• plsdof
• The three-pass regression filter: a new approach to forecasting using many predictors
• Methodology and theory for partial least squares applied to functional data
• A new universal resample-stable bootstrap-based stopping criterion for PLS component construction
• Functional Principal Component Regression and Functional Partial Least‐squares Regression: An Overview and a Comparative Study
• Determining the number of components in PLS regression on incomplete data set
• Computing Fréchet derivatives in partial least squares regression
• Sparse functional partial least squares regression with a locally sparse slope function

This page was built for publication: The Degrees of Freedom of Partial Least Squares Regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q86894)