Regularization of case-specific parameters for robustness and efficiency
From MaRDI portal
Abstract: Regularization methods allow one to handle a variety of inferential problems where there are more covariates than cases. This allows one to consider a potentially enormous number of covariates for a problem. We exploit the power of these techniques, supersaturating models by augmenting the "natural" covariates in the problem with an additional indicator for each case in the data set. We attach a penalty term for these case-specific indicators which is designed to produce a desired effect. For regression methods with squared error loss, an penalty produces a regression which is robust to outliers and high leverage cases; for quantile regression methods, an penalty decreases the variance of the fit enough to overcome an increase in bias. The paradigm thus allows us to robustify procedures which lack robustness and to increase the efficiency of procedures which are robust. We provide a general framework for the inclusion of case-specific parameters in regularization problems, describing the impact on the effective loss for a variety of regression and classification problems. We outline a computational strategy by which existing software can be modified to solve the augmented regularization problem, providing conditions under which such modification will converge to the optimum solution. We illustrate the benefits of including case-specific parameters in the context of mean regression and quantile regression through analysis of NHANES and linguistic data sets.
Recommendations
- Regularization in statistics
- Robust sparse regression and tuning parameter selection via the efficient bootstrap information criteria
- Bayesian regularized quantile regression
- Regularization of linear regression problems
- Fully efficient robust estimation, outlier detection, and variable selection via penalized regression
Cites work
- scientific article; zbMATH DE number 5957245 (Why is no real title available?)
- scientific article; zbMATH DE number 45848 (Why is no real title available?)
- scientific article; zbMATH DE number 47282 (Why is no real title available?)
- scientific article; zbMATH DE number 47310 (Why is no real title available?)
- scientific article; zbMATH DE number 1266748 (Why is no real title available?)
- scientific article; zbMATH DE number 1332320 (Why is no real title available?)
- scientific article; zbMATH DE number 775727 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- 10.1162/153244303322753643
- A Nonparametric Regression Approach to Syringe Grading for Quality Improvement
- A decision-theoretic generalization of on-line learning and an application to boosting
- Applied Linear Regression
- Asymmetric Least Squares Estimation and Testing
- Bayesian synthesis: combining subjective analyses, with an application to ozone data
- Convexity, Classification, and Risk Bounds
- Does modeling lead to more accurate classification? A study of relative efficiency in linear classification
- Group variable selection via a hierarchical lasso and its oracle property
- Least angle regression. (With discussion)
- Limiting distributions for \(L_1\) regression estimators under general conditions
- Model Selection and Estimation in Regression with Grouped Variables
- On ψ-Learning
- Piecewise linear regularized solution paths
- Quantile regression.
- Regression Quantiles
- Regularization and Variable Selection Via the Elastic Net
- Regularization of case-specific parameters for robustness and efficiency
- Ridge Regression: Biased Estimation for Nonorthogonal Problems
- Robust Statistics
- Robust Truncated Hinge Loss Support Vector Machines
- Robustness and regularization of support vector machines
- SSVM: A smooth support vector machine for classification
- The Gaussian hare and the Laplacian tortoise: computability of squared-error versus absolute-error estimators. With comments by Ronald A. Thisted and M. R. Osborne and a rejoinder by the authors
Cited in
(26)- Robust regression with compositional covariates
- Efficient information-based criteria for model selection in quantile regression
- A Compressive Sensing Based Analysis of Anomalies in Generalized Linear Models
- Regression with outlier shrinkage
- Robust regression for highly corrupted response by shifting outliers
- Robust censored regression with \(\ell_1\)-norm regularization
- High-dimensional inference robust to outliers with ℓ1-norm penalization
- Inference robust to outliers with \(\ell_1\)-norm penalization
- Robust logistic regression with shift parameter estimation
- Modified check loss for efficient estimation via model selection in quantile regression
- Classification of histogram-valued data with support histogram machines
- Quantile regression feature selection and estimation with grouped variables using Huber approximation
- Robust estimation and outlier detection for varying-coefficient models via penalized regression
- Prediction-based regularization using data augmented regression
- A tailored multivariate mixture model for detecting proteins of concordant change among virulent strains of \textit{clostridium perfringens}
- Regularization of case-specific parameters for robustness and efficiency
- Efficient quantile regression for heteroscedastic models
- Robust linear regression: A review and comparison
- Nonlinear regression models for heterogeneous data with massive outliers
- Rapid penalized likelihood-based outlier detection via heteroskedasticity test
- Robust MAVE through nonconvex penalized regression
- Simultaneous feature selection and outlier detection with optimality guarantees
- scientific article; zbMATH DE number 5274762 (Why is no real title available?)
- Individual Transition Label Noise Logistic Regression in Binary Classification for Incorrectly Labeled Data
- Minimizing Sum of Truncated Convex Functions and Its Applications
- Outlier detection and robust mixture modeling using nonconvex penalized likelihood
This page was built for publication: Regularization of case-specific parameters for robustness and efficiency
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q252778)
