The influence function of penalized regression estimators

DOI10.1080/02331888.2014.922563MaRDI QIDQ3462119zbMATH OpenOpenAlexFDO

Authors Viktoria Öllerer, Christophe Croux, Andreas Alfons

Publication date 4 January 2016

Published in Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1501.01208, https://lirias.kuleuven.be/handle/123456789/425563

zbMATH Keywords

influence functionlassoleast trimmed squarespenalized M-regressionsparseness

Mathematics Subject Classification ID

Ridge regression; shrinkage estimators (Lasso) (62J07) Diagnostics, and linear inference and regression (62J20)

Abstract: To perform regression analysis in high dimensions, lasso or ridge estimation are a common choice. However, it has been shown that these methods are not robust to outliers. Therefore, alternatives as penalized M-estimation or the sparse least trimmed squares (LTS) estimator have been proposed. The robustness of these regression methods can be measured with the influence function. It quantifies the effect of infinitesimal perturbations in the data. Furthermore it can be used to compute the asymptotic variance and the mean squared error. In this paper we compute the influence function, the asymptotic variance and the mean squared error for penalized M-estimators and the sparse LTS estimator. The asymptotic biasedness of the estimators make the calculations nonstandard. We show that only M-estimators with a loss function with a bounded derivative are robust against regression outliers. In particular, the lasso has an unbounded influence function.

Recommendations

• Influence functions for penalized M-estimators
• Robust sparse regression with high-breakdown value
• Sparse least trimmed squares regression for analyzing high-dimensional large data sets
• Robust and sparse estimators for linear regression models
• Penalised robust estimators for sparse and high-dimensional linear models

Cites work

• Convergence of a block coordinate descent method for nondifferentiable minimization
• scientific article; zbMATH DE number 3915531 (Why is no real title available?)
• scientific article; zbMATH DE number 192992 (Why is no real title available?)
• Pathwise coordinate optimization
• Robust Statistics
• Robust Statistics
• Sparse least trimmed squares regression for analyzing high-dimensional large data sets
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Weighted LAD-LASSO method for robust parameter estimation and variable selection in regression

Cited in

(15)

• Robust and sparse estimators for linear regression models
• Regularization parameter selection for penalized empirical likelihood estimator
• Sparse least trimmed squares regression for analyzing high-dimensional large data sets
• Asymptotic linear expansion of regularized M-estimators
• Detangling robustness in high dimensions: composite versus model-averaged estimation
• Influence functions for penalized M-estimators
• Gaining Outlier Resistance With Progressive Quantiles: Fast Algorithms and Theoretical Studies
• The influence function of semiparametric two-step estimators with estimated control variables
• Robust estimation in regression and classification methods for large dimensional data
• Least sum of squares of trimmed residuals regression
• Robust and sparse logistic regression
• Robust variable selection in high-dimensional nonparametric additive model
• Robust variable selection criteria for the penalized regression
• Robust variable selection of linear model under BAR penalty
• Influence functions for LTS and reweighted LTS estimators based on moment conditions

This page was built for publication: The influence function of penalized regression estimators

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3462119)