Characterization of the equivalence of robustification and regularization in linear and matrix regression
From MaRDI portal
(Redirected from Publication:723995)
Factor analysis and principal components; correspondence analysis (62H25) Learning and adaptive systems in artificial intelligence (68T05) Convex programming (90C25) Estimation in multivariate analysis (62H12) Ridge regression; shrinkage estimators (Lasso) (62J07) Robustness and adaptive procedures (parametric inference) (62F35)
Abstract: The notion of developing statistical methods in machine learning which are robust to adversarial perturbations in the underlying data has been the subject of increasing interest in recent years. A common feature of this work is that the adversarial robustification often corresponds exactly to regularization methods which appear as a loss function plus a penalty. In this paper we deepen and extend the understanding of the connection between robustification and regularization (as achieved by penalization) in regression problems. Specifically, (a) in the context of linear regression, we characterize precisely under which conditions on the model of uncertainty used and on the loss function penalties robustification and regularization are equivalent, and (b) we extend the characterization of robustification and regularization to matrix regression problems (matrix completion and Principal Component Analysis).
Recommendations
- The generalized equivalence of regularization and min-max robustification in linear mixed models
- Robustness and regularization of support vector machines
- Robust regression via mutivariate regression depth
- Robust regression using biased objectives
- Robust classification and regression using support vector machines
Cites work
- scientific article; zbMATH DE number 194744 (Why is no real title available?)
- scientific article; zbMATH DE number 2107836 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- scientific article; zbMATH DE number 6125590 (Why is no real title available?)
- A survey of robust statistics
- Adaptive robust variable selection
- An Analysis of the Total Least Squares Problem
- Consistency of the structured total least squares estimator in a multivariate errors-in-variables model
- Convex analysis and monotone operator theory in Hilbert spaces
- Data-driven robust optimization
- Elastic-net regularization in learning theory
- Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization
- High breakdown estimators for principal components: the projection-pursuit approach revis\-ited
- High-breakdown robust multivariate methods
- Least Median of Squares Regression
- Lipschitz behavior of the robust regularization
- Measurement Error in Nonlinear Models
- On combining machine learning with decision making
- Oracle-based robust optimization via online learning
- Overview of total least-squares methods
- Penalized composite quasi-likelihood for ultrahigh dimensional variable selection
- Principal Components Analysis Based on Multivariate MM Estimators With Fast and Robust Bootstrap
- Regularization and Variable Selection Via the Elastic Net
- Robust Regression and Lasso
- Robust Solutions to Least-Squares Problems with Uncertain Data
- Robust Statistics
- Robust optimization
- Robust principal component analysis?
- Robust regression: Asymptotics, conjectures and Monte Carlo
- The Influence Curve and Its Role in Robust Estimation
- Theory and applications of robust optimization
Cited in
(35)- Robust linear algebra
- Certifiably optimal sparse principal component analysis
- Rejoinder: ``Sparse regression: scalable algorithms and empirical performance
- An Analytical and Geometric Perspective on Adversarial Robustness
- Sparse classification: a scalable discrete optimization perspective
- Regularization via mass transportation
- Sparse regression for large data sets with outliers
- A look at robustness and stability of \(\ell_1\)-versus \(\ell_0\)-regularization: discussion of papers by Bertsimas et al. and Hastie et al.
- Sparse regression over clusters: SparClur
- An Alternating Method for Cardinality-Constrained Optimization: A Computational Study for the Best Subset Selection and Sparse Portfolio Problems
- Certifiably optimal sparse inverse covariance estimation
- Optimization problems for machine learning: a survey
- Robust prediction of domain compositions from uncertain data using isometric logratio transformations in a penalized multivariate Fay–Herriot model
- The backbone method for ultra-high dimensional sparse machine learning
- Calibration of distributionally robust empirical optimization models
- Detecting racial bias in jury selection
- OR forum: An algorithmic approach to linear regression
- Robust newsvendor problem with autoregressive demand
- Robust Linear Regression via $\ell_0$ Regularization
- Robust grouped variable selection using distributionally robust optimization
- The generalized equivalence of regularization and min-max robustification in linear mixed models
- scientific article; zbMATH DE number 7306912 (Why is no real title available?)
- Robust \(\mathcal S\)-regularity of matrix pencils applied to the analysis of descriptor models
- Sparse hierarchical regression with polynomials
- A robust learning approach for regression models based on distributionally robust optimization
- Scalable holistic linear regression
- Diametrical risk minimization: theory and computations
- Near-optimal nonlinear regression trees
- Distributionally robust optimization. A review on theory and applications
- Ambulance emergency response optimization in developing countries
- Adversarial classification via distributional robustness with Wasserstein ambiguity
- Learning sparse nonlinear dynamics via mixed-integer optimization
- RSG: Beating Subgradient Method without Smoothness and Strong Convexity
- Robust Wasserstein profile inference and applications to machine learning
- An elastic net penalized small area model combining unit- and area-level data for regional hypertension prevalence estimation
This page was built for publication: Characterization of the equivalence of robustification and regularization in linear and matrix regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q723995)
