Regularization of case-specific parameters for robustness and efficiency
From MaRDI portal
Abstract: Regularization methods allow one to handle a variety of inferential problems where there are more covariates than cases. This allows one to consider a potentially enormous number of covariates for a problem. We exploit the power of these techniques, supersaturating models by augmenting the "natural" covariates in the problem with an additional indicator for each case in the data set. We attach a penalty term for these case-specific indicators which is designed to produce a desired effect. For regression methods with squared error loss, an penalty produces a regression which is robust to outliers and high leverage cases; for quantile regression methods, an penalty decreases the variance of the fit enough to overcome an increase in bias. The paradigm thus allows us to robustify procedures which lack robustness and to increase the efficiency of procedures which are robust. We provide a general framework for the inclusion of case-specific parameters in regularization problems, describing the impact on the effective loss for a variety of regression and classification problems. We outline a computational strategy by which existing software can be modified to solve the augmented regularization problem, providing conditions under which such modification will converge to the optimum solution. We illustrate the benefits of including case-specific parameters in the context of mean regression and quantile regression through analysis of NHANES and linguistic data sets.
Recommendations
- Regularization in statistics
- Robust sparse regression and tuning parameter selection via the efficient bootstrap information criteria
- Bayesian regularized quantile regression
- Regularization of linear regression problems
- Fully efficient robust estimation, outlier detection, and variable selection via penalized regression
Cites work
- 10.1162/153244303322753643
- A decision-theoretic generalization of on-line learning and an application to boosting
- A Nonparametric Regression Approach to Syringe Grading for Quality Improvement
- Applied Linear Regression
- Asymmetric Least Squares Estimation and Testing
- Bayesian synthesis: combining subjective analyses, with an application to ozone data
- Convexity, Classification, and Risk Bounds
- Does modeling lead to more accurate classification? A study of relative efficiency in linear classification
- Group variable selection via a hierarchical lasso and its oracle property
- scientific article; zbMATH DE number 5957245 (Why is no real title available?)
- scientific article; zbMATH DE number 45848 (Why is no real title available?)
- scientific article; zbMATH DE number 47282 (Why is no real title available?)
- scientific article; zbMATH DE number 47310 (Why is no real title available?)
- scientific article; zbMATH DE number 1266748 (Why is no real title available?)
- scientific article; zbMATH DE number 1332320 (Why is no real title available?)
- scientific article; zbMATH DE number 775727 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- Least angle regression. (With discussion)
- Limiting distributions for \(L_1\) regression estimators under general conditions
- Model Selection and Estimation in Regression with Grouped Variables
- On ψ-Learning
- Piecewise linear regularized solution paths
- Quantile regression.
- Regression Quantiles
- Regularization and Variable Selection Via the Elastic Net
- Regularization of case-specific parameters for robustness and efficiency
- Ridge Regression: Biased Estimation for Nonorthogonal Problems
- Robust Statistics
- Robust Truncated Hinge Loss Support Vector Machines
- Robustness and regularization of support vector machines
- SSVM: A smooth support vector machine for classification
- The Gaussian hare and the Laplacian tortoise: computability of squared-error versus absolute-error estimators. With comments by Ronald A. Thisted and M. R. Osborne and a rejoinder by the authors
Cited in
(28)- Quantile regression feature selection and estimation with grouped variables using Huber approximation
- Efficient information-based criteria for model selection in quantile regression
- Robust MAVE through nonconvex penalized regression
- Robust regression with compositional covariates
- Regularization of case-specific parameters for robustness and efficiency
- A Compressive Sensing Based Analysis of Anomalies in Generalized Linear Models
- Minimizing Sum of Truncated Convex Functions and Its Applications
- Regression with outlier shrinkage
- Robust linear regression: A review and comparison
- A tailored multivariate mixture model for detecting proteins of concordant change among virulent strains of \textit{clostridium perfringens}
- Robust regression for highly corrupted response by shifting outliers
- Outlier detection and robust mixture modeling using nonconvex penalized likelihood
- Nonlinear regression models for heterogeneous data with massive outliers
- Robust estimation and outlier detection for varying-coefficient models via penalized regression
- Rapid penalized likelihood-based outlier detection via heteroskedasticity test
- Inference robust to outliers with \(\ell_1\)-norm penalization
- Efficient quantile regression for heteroscedastic models
- scientific article; zbMATH DE number 5274762 (Why is no real title available?)
- Modified check loss for efficient estimation via model selection in quantile regression
- Simultaneous feature selection and outlier detection with optimality guarantees
- Robust censored regression with _1-norm regularization
- Robust logistic regression with shift parameter estimation
- Classification of histogram-valued data with support histogram machines
- High-dimensional inference robust to outliers with ℓ1-norm penalization
- Individual Transition Label Noise Logistic Regression in Binary Classification for Incorrectly Labeled Data
- The adaptive robust and efficient variable selection method for the linear regression model
- Distributed learning with adversarial contamination: an efficient and optimal approach based on dynamic regularization
- Prediction-based regularization using data augmented regression
This page was built for publication: Regularization of case-specific parameters for robustness and efficiency
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q252778)
