Characterization of the equivalence of robustification and regularization in linear and matrix regression
From MaRDI portal
Robustness and adaptive procedures (parametric inference) (62F35) Estimation in multivariate analysis (62H12) Factor analysis and principal components; correspondence analysis (62H25) Ridge regression; shrinkage estimators (Lasso) (62J07) Learning and adaptive systems in artificial intelligence (68T05) Convex programming (90C25)
Abstract: The notion of developing statistical methods in machine learning which are robust to adversarial perturbations in the underlying data has been the subject of increasing interest in recent years. A common feature of this work is that the adversarial robustification often corresponds exactly to regularization methods which appear as a loss function plus a penalty. In this paper we deepen and extend the understanding of the connection between robustification and regularization (as achieved by penalization) in regression problems. Specifically, (a) in the context of linear regression, we characterize precisely under which conditions on the model of uncertainty used and on the loss function penalties robustification and regularization are equivalent, and (b) we extend the characterization of robustification and regularization to matrix regression problems (matrix completion and Principal Component Analysis).
Recommendations
- The generalized equivalence of regularization and min-max robustification in linear mixed models
- Robustness and regularization of support vector machines
- Robust regression via mutivariate regression depth
- Robust regression using biased objectives
- Robust classification and regression using support vector machines
Cites work
- A survey of robust statistics
- Adaptive robust variable selection
- An Analysis of the Total Least Squares Problem
- Consistency of the structured total least squares estimator in a multivariate errors-in-variables model
- Convex analysis and monotone operator theory in Hilbert spaces
- Data-driven robust optimization
- Elastic-net regularization in learning theory
- Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization
- High breakdown estimators for principal components: the projection-pursuit approach revis\-ited
- High-breakdown robust multivariate methods
- scientific article; zbMATH DE number 194744 (Why is no real title available?)
- scientific article; zbMATH DE number 2107836 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- scientific article; zbMATH DE number 6125590 (Why is no real title available?)
- Least Median of Squares Regression
- Lipschitz behavior of the robust regularization
- Measurement Error in Nonlinear Models
- On combining machine learning with decision making
- Oracle-based robust optimization via online learning
- Overview of total least-squares methods
- Penalized composite quasi-likelihood for ultrahigh dimensional variable selection
- Principal Components Analysis Based on Multivariate MM Estimators With Fast and Robust Bootstrap
- Regularization and Variable Selection Via the Elastic Net
- Robust optimization
- Robust principal component analysis?
- Robust Regression and Lasso
- Robust regression: Asymptotics, conjectures and Monte Carlo
- Robust Solutions to Least-Squares Problems with Uncertain Data
- Robust Statistics
- The Influence Curve and Its Role in Robust Estimation
- Theory and applications of robust optimization
Cited in
(38)- Optimization problems for machine learning: a survey
- The generalized equivalence of regularization and min-max robustification in linear mixed models
- Sparse classification: a scalable discrete optimization perspective
- Distributionally robust optimization. A review on theory and applications
- Sparse regression over clusters: SparClur
- Robust grouped variable selection using distributionally robust optimization
- The backbone method for ultra-high dimensional sparse machine learning
- Detecting racial bias in jury selection
- Scalable holistic linear regression
- Certifiably optimal sparse inverse covariance estimation
- A look at robustness and stability of \(\ell_1\)-versus \(\ell_0\)-regularization: discussion of papers by Bertsimas et al. and Hastie et al.
- Rejoinder: ``Sparse regression: scalable algorithms and empirical performance
- Sparse regression for large data sets with outliers
- Certifiably optimal sparse principal component analysis
- Near-optimal nonlinear regression trees
- Adversarial classification via distributional robustness with Wasserstein ambiguity
- OR forum: An algorithmic approach to linear regression
- Robust newsvendor problem with autoregressive demand
- RSG: Beating Subgradient Method without Smoothness and Strong Convexity
- A robust learning approach for regression models based on distributionally robust optimization
- Robust Linear Regression via \ell₀ Regularization
- Calibration of distributionally robust empirical optimization models
- An Alternating Method for Cardinality-Constrained Optimization: A Computational Study for the Best Subset Selection and Sparse Portfolio Problems
- Ambulance emergency response optimization in developing countries
- scientific article; zbMATH DE number 7306912 (Why is no real title available?)
- Regularization via mass transportation
- Robust Wasserstein profile inference and applications to machine learning
- Robust \(\mathcal S\)-regularity of matrix pencils applied to the analysis of descriptor models
- An elastic net penalized small area model combining unit- and area-level data for regional hypertension prevalence estimation
- An Analytical and Geometric Perspective on Adversarial Robustness
- Learning sparse nonlinear dynamics via mixed-integer optimization
- Robust prediction of domain compositions from uncertain data using isometric logratio transformations in a penalized multivariate Fay–Herriot model
- Diametrical risk minimization: theory and computations
- Robust linear algebra
- On the complexity of p-order cone programs
- Low-rank matrix estimation via nonconvex spectral regularized methods in errors-in-variables matrix regression
- The distributionally robust prediction error of the \(\sqrt{\mathrm{LASSO}}\) and related estimators
- Sparse hierarchical regression with polynomials
This page was built for publication: Characterization of the equivalence of robustification and regularization in linear and matrix regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q723995)
