Anomalies in the Foundations of Ridge Regression
From MaRDI portal
Publication:3182012
Abstract: Anomalies persist in the foundations of ridge regression as set forth in Hoerl and Kennard (1970) and subsequently. Conventional ridge estimators and their properties do not follow on constraining lengths of solution vectors using LaGrange's method, as claimed. Estimators so constrained have singular distributions; the proposed solutions are not necessarily minimizing; and heretofore undiscovered bounds are exhibited for the ridge parameter. None of the considerable literature on estimation, prediction, cross--validation, choice of ridge parameter, and related issues, collectively known as ridge regression, is consistent with constrained optimization, nor with corresponding inequality constraints. The problem is traced to a misapplication of LaGrange's principle, failure to recognize the singularity of distributions, and misplaced links between constraints and the ridge parameter. Other principles, based on condition numbers, are seen to validate both conventional ridge and surrogate ridge regression to be defined. Numerical studies illustrate that ridge analysis often exhibits some of the same pathologies it is intended to redress.
Recommendations
Cites work
- scientific article; zbMATH DE number 3753890 (Why is no real title available?)
- scientific article; zbMATH DE number 3323651 (Why is no real title available?)
- "Ridge Analysis" of Response Surfaces
- A Biometrics Invited Paper. The Analysis and Selection of Variables in Linear Regression
- A Simulation Study of Alternatives to Ordinary Least Squares
- Characterization of Ridge Trace Behavior
- Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter
- Generalized Inverses, Ridge Regression, Biased Linear Estimation, and Nonlinear Estimation
- Inequalities: theory of majorization and its applications
- Matrix Analysis
- Minimax linear, ridge and shrunken estimators for linear parameters
- Ridge Regression in Practice
- Ridge Regression: Biased Estimation for Nonorthogonal Problems
- Ridge regression:some simulations
- Solving Systems of Linear Equations With a Positive Definite, Symmetric, but Possibly Ill-Conditioned Matrix
- Some Comments on C P
- The Acceptability of Regression Solutions: Another Look at Computational Accuracy
- The Effect of Errors in the Independent Variables in Linear Regression
- The Relationship between Variable Selection and Data Agumentation and a Method for Prediction
- The generalized ridge estimator and improved adjustments for regression parameters
- Tolerance and Condition in Regression Computations
Cited in
(17)- The coefficient of determination in the ridge regression
- A note about the corrected VIF
- On mitigating collinearity through mixtures
- Revision: variance inflation in regression
- Mitigating collinearity in linear regression models using ridge, surrogate and raised estimators
- On the practice of rescaling covariates
- Anomalies in the foundations of ridge regression: some clarifications
- Enhanced ridge regressions
- Collinearity: revisiting the variance inflation factor in ridge regression
- Nonparametric principal components regression
- Concentration reversals in ridge regression
- Designs enhancing Fisher information
- Confronting collinearity in environmental regression models: evidence from world data
- The red indicator and corrected VIFs in generalized linear models
- A Note on Collinearity Diagnostics and Centering
- Standardization of Variables and Collinearity Diagnostic in Ridge Regression
- Penalized regression with ordinal predictors
This page was built for publication: Anomalies in the Foundations of Ridge Regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3182012)
