A modelling framework for regression with collinearity

DOI10.1016/J.JSPI.2023.07.001MaRDI QIDQ6049416zbMATH OpenFDO

Authors Takeaki Kariya, Hiroshi Kurata, Takaki Hayashi

Publication date 15 September 2023

Published in Journal of Statistical Planning and Inference (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://arxiv.org/abs/2301.03015

zbMATH Keywords

collinearity effectempirically effective modellingmodel selection processOLS

Mathematics Subject Classification ID

Linear regression; mixed models (62J05) Diagnostics, and linear inference and regression (62J20)

Abstract: This study addresses a fundamental, yet overlooked, gap between standard theory and empirical modelling practices in the OLS regression model $$ with collinearity. In fact, while an estimated model in practice is desired to have stability and efficiency in its "individual OLS estimates", $$ itself has no capacity to identify and control the collinearity in $$ and hence no theory including model selection process (MSP) would fill this gap unless $$ is controlled in view of sampling theory. In this paper, first introducing a new concept of "empirically effective modelling" (EEM), we propose our EEM methodology (EEM-M) as an integrated process of two MSPs with data $$ given. The first MSP uses $$ only, called the XMSP, and pre-selects a class $scrD$ of models with individually inefficiency-controlled and collinearity-controlled OLS estimates, where the corresponding two controlling variables are chosen from predictive standard error of each estimate. Next, defining an inefficiency-collinearity risk index for each model, a partial ordering is introduced onto the set of models to compare without using $$, where the better-ness and admissibility of models are discussed. The second MSP is a commonly used MSP that uses $$, and evaluates total model performance as a whole by such AIC, BIC, etc. to select an optimal model from $scrD$. Third, to materialize the XMSP, two algorithms are proposed.

Recommendations

• Multiple choice from competing regression models under multicollinearity based on standardized update
• Bayesian method for solving the problem of multicollinearity in regression
• scientific article; zbMATH DE number 807597
• scientific article; zbMATH DE number 1051708

Cites work

• Collinearity and least squares regression
• Estimable group effects for strongly correlated variables in linear models
• scientific article; zbMATH DE number 49700 (Why is no real title available?)
• scientific article; zbMATH DE number 3254517 (Why is no real title available?)
• Improved Empirical Bayes Ridge Regression Estimators Under Multicollinearity
• Number of predictors and multicollinearity: What are their effects on error and bias in regression?
• Principal component analysis.
• Regression analysis by example
• Regularization and Variable Selection Via the Elastic Net
• Testing Statistical Hypotheses
• Theory of Preliminary Test and Stein‐Type Estimation With Applications

Cited in

(3)

• Multiple choice from competing regression models under multicollinearity based on standardized update
• Confronting collinearity in environmental regression models: evidence from world data
• Ineffectiveness of model selection via t-test in regression with collinearity

This page was built for publication: A modelling framework for regression with collinearity

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6049416)