Outlier detection using nonconvex penalized regression

DOI10.1198/JASA.2011.TM10390zbMATH Open1232.62068arXiv1006.2592OpenAlexW1969515697MaRDI QIDQ3095181FDOQ3095181

Authors: Yiyuan She, Art B. Owen 

Publication date: 28 October 2011

Published in: Journal of the American Statistical Association (Search for Journal in Brave)

Abstract: This paper studies the outlier detection problem from the point of view of penalized regressions. Our regression model adds one mean shift parameter for each of the $n$ data points. We then apply a regularization favoring a sparse vector of mean shift parameters. The usual $L_1$ penalty yields a convex criterion, but we find that it fails to deliver a robust estimator. The $L_1$ penalty corresponds to soft thresholding. We introduce a thresholding (denoted by $Theta$) based iterative procedure for outlier detection ($Theta$-IPOD). A version based on hard thresholding correctly identifies outliers on some hard test problems. We find that $Theta$-IPOD is much faster than iteratively reweighted least squares for large data because each iteration costs at most $O(np)$ (and sometimes much less) avoiding an $O(n{p}^2)$ least squares estimate. We describe the connection between $Theta$-IPOD and $M$-estimators. Our proposed method has one tuning parameter with which to both identify outliers and estimate regression coefficients. A data-dependent choice can be made based on BIC. The tuned $Theta$-IPOD shows outstanding performance in identifying outliers in various situations in comparison to other existing approaches. This methodology extends to high-dimensional modeling with $pggn$, if both the coefficient vector and the outlier pattern are sparse.

Full work available at URL: https://arxiv.org/abs/1006.2592

Recommendations

• Fully efficient robust estimation, outlier detection, and variable selection via penalized regression
• Iterative gradient descent for outlier detection
• Outlier detection and robust variable selection via the penalized weighted LAD-LASSO method
• Inference robust to outliers with \(\ell_1\)-norm penalization
• Large-scale regression with non-convex loss and penalty

zbMATH Keywords

robust regressionsparsitythresholding\(M\)-estimation

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Nonparametric regression and quantile regression (62G08) Nonparametric robustness (62G35)

Cited In (99)

• Regression with outlier shrinkage
• The main contributions of robust statistics to statistical science and a new challenge
• Robust ridge M-estimators with pretest and Stein-rule shrinkage for an intercept term
• Robust difference-based outlier detection
• ORCA: outlier detection and robust clustering for attributed graphs
• Simultaneous feature selection and outlier detection with optimality guarantees
• A New Principle for Tuning-Free Huber Regression
• Minimizing a sum of clipped convex functions
• Title not available (Why is that?)
• Gaining Outlier Resistance With Progressive Quantiles: Fast Algorithms and Theoretical Studies
• Robust finite mixture regression for heterogeneous targets
• Valid Inference Corrected for Outlier Removal
• A general method for robust Bayesian modeling
• Outlier detection using difference-based variance estimators in multiple regression
• Title not available (Why is that?)
• Fully efficient robust estimation, outlier detection, and variable selection via penalized regression
• Robust MAVE through nonconvex penalized regression
• Penalised robust estimators for sparse and high-dimensional linear models
• Robust reduced-rank modeling via rank regression
• Mean-shift outliers model in skew scale-mixtures of normal distributions
• Computational Outlier Detection Methods in Sliced Inverse Regression
• Outlier detection under a covariate-adjusted exponential regression model with censored data
• Inference robust to outliers with ℓ₁-norm penalization
• An iterative algorithm for fitting nonconvex penalized generalized linear models with grouped predictors
• Adaptive and robust multi-task learning
• Multiple outliers detection in sparse high-dimensional regression
• Robust estimation of models for longitudinal data with dropouts and outliers
• A refined convergence analysis of \(\mathrm{pDCA}_{e}\) with applications to simultaneous sparse recovery and outlier detection
• Robust high-dimensional regression for data with anomalous responses
• Asymptotically Exact Data Augmentation: Models, Properties, and Algorithms
• Rapid penalized likelihood-based outlier detection via heteroskedasticity test
• Penalized unsupervised learning with outliers
• Low-rank elastic-net regularized multivariate Huber regression model
• High-dimensional inference robust to outliers with ℓ1-norm penalization
• Title not available (Why is that?)
• Evaluating visual properties via robust HodgeRank
• Outlier detection and least trimmed squares approximation using semi-definite programming
• A novel robust approach for analysis of longitudinal data
• Robust and Accurate Inference via a Mixture of Gaussian and Student’s t Errors
• Perspective maximum likelihood-type estimation via proximal decomposition
• Robust estimation and outlier detection for varying-coefficient models via penalized regression
• Robust Sparse Regression with High-Breakdown Value
• Outlier detection and robust mixture modeling using nonconvex penalized likelihood
• Consistency of the elastic net under a finite second moment assumption on the noise
• The solution path of the generalized lasso
• A Tailored Multivariate Mixture Model for Detecting Proteins of Concordant Change Among Virulent Strains of Clostridium Perfringens
• Provable training set debugging for linear regression
• Robust subset selection
• Robust linear regression: A review and comparison
• An adversarial optimization approach to efficient outlier removal
• Lasso ANOVA decompositions for matrix and tensor data
• A Framework of Learning Through Empirical Gain Maximization
• Iterative gradient descent for outlier detection
• Conditional selective inference for robust regression and outlier detection using piecewise-linear homotopy continuation
• Linear regression with sparsely permuted data
• Multiple hypothesis testing adjusted for latent variables, with an application to the AGEMAP gene expression data
• Sparse least trimmed squares regression for analyzing high-dimensional large data sets
• The Mean-shift Outlier Model under Skew Normal Distribution
• Computational and statistical analyses for robust non-convex sparse regularized regression problem
• High-dimensional heteroscedastic regression with an application to eQTL data analysis
• Sparse Reduced Rank Huber Regression in High Dimensions
• Minimizing Sum of Truncated Convex Functions and Its Applications
• Robust moderately clipped LASSO for simultaneous outlier detection and variable selection
• Robust regression with compositional covariates
• Decomposition into low-rank plus additive matrices for background/foreground separation: a review for a comparative evaluation with a large-scale dataset
• Penalized MM regression estimation withL_γpenalty: a robust version of bridge regression
• A high breakdown, high efficiency and bounded influence modified GM estimator based on support vector regression
• Exponential regression for censored data with outliers
• Simultaneous outlier detection and variable selection for spatial Durbin model
• Letter to the editor
• Regression with linked datasets subject to linkage error
• Trustworthy regularized huber regression for outlier detection
• Robust logistic regression with shift parameter estimation
• Robust variable selection and estimation via adaptive elastic net S-estimators for linear regression
• Robust and tuning-free sparse linear regression via square-root slope
• Title not available (Why is that?)
• Robust estimation of covariance matrices: adversarial contamination and beyond
• Robust regression using probabilistically linked data
• Robust statistics: a selective overview and new directions
• Outlier detection in statistical modeling via multivariate adaptive regression splines
• Outlier detection and robust variable selection via the penalized weighted LAD-LASSO method
• Adaptive efficient and double-robust regression based on generalized empirical likelihood
• Using History Matching for Prior Choice
• CR-Lasso: robust cellwise regularized sparse regression
• A robust support vector machine for labeling errors
• Influence Diagnostics for Correlated Binomial Regression Models: An Application to a Data Set on High-Cost Health Services Occurrence
• Simultaneous spatial smoothing and outlier detection using penalized regression, with application to childhood obesity surveillance from electronic health records
• A Sharper Computational Tool for Regression
• Nonlinear regression models for heterogeneous data with massive outliers
• A Bayesian framework for robust quantitative trait locus mapping and outlier detection
• Title not available (Why is that?)
• LPNN-based approach for LASSO problem via a sequence of regularized minimizations
• Influence Diagnostics for High-Dimensional Lasso Regression
• Sparse local influence analysis
• A User-Friendly Computational Framework for Robust Structured Regression with the L₂ Criterion
• Multiple Influential Point Detection in High Dimensional Regression Spaces
• Robust censored regression with \(\ell_1\)-norm regularization
• A new correction approach for information criteria to detect outliers in regression modeling
• A robust high dimensional estimation of a finite mixture of the generalized linear model

Uses Software

• robustbase

This page was built for publication: Outlier detection using nonconvex penalized regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3095181)