Learning models with uniform performance via distributionally robust optimization
From MaRDI portal
Publication:820804
Abstract: A common goal in statistics and machine learning is to learn models that can perform well against distributional shifts, such as latent heterogeneous subpopulations, unknown covariate shifts, or unmodeled temporal effects. We develop and analyze a distributionally robust stochastic optimization (DRO) framework that learns a model providing good performance against perturbations to the data-generating distribution. We give a convex formulation for the problem, providing several convergence guarantees. We prove finite-sample minimax upper and lower bounds, showing that distributional robustness sometimes comes at a cost in convergence rates. We give limit theorems for the learned parameters, where we fully specify the limiting distribution so that confidence intervals can be computed. On real tasks including generalizing to unknown subpopulations, fine-grained recognition, and providing good tail performance, the distributionally robust approach often exhibits improved performance.
Recommendations
- scientific article; zbMATH DE number 7668294
- A robust learning approach for regression models based on distributionally robust optimization
- A stochastic subgradient method for distributionally robust non-convex and non-smooth learning
- Recovering best statistical guarantees via the empirical divergence-based distributionally robust optimization
- Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations
Cites work
- scientific article; zbMATH DE number 5547839 (Why is no real title available?)
- scientific article; zbMATH DE number 3911472 (Why is no real title available?)
- scientific article; zbMATH DE number 3921733 (Why is no real title available?)
- scientific article; zbMATH DE number 49190 (Why is no real title available?)
- scientific article; zbMATH DE number 1795125 (Why is no real title available?)
- scientific article; zbMATH DE number 2107836 (Why is no real title available?)
- scientific article; zbMATH DE number 7246284 (Why is no real title available?)
- scientific article; zbMATH DE number 3252891 (Why is no real title available?)
- scientific article; zbMATH DE number 3322635 (Why is no real title available?)
- A data-driven software tool for enabling cooperative information sharing among police departments
- A framework for optimization under ambiguity
- A new look at independence
- A theory of learning from different domains
- Ambiguity in portfolio selection
- Asymptotic Statistics
- Asymptotics in statistics. Some basic concepts.
- Classifier technology and the illusion of progress
- Coherent measures of risk
- Concentration inequalities. A nonasymptotic theory of independence
- Confidence Intervals for Maximin Effects in Inhomogeneous Large-Scale Data
- Covariate shift adaptation by importance weighted cross validation
- Data-driven chance constrained stochastic program
- Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations
- Data-driven robust optimization
- Distributionally robust optimization under moment uncertainty with application to data-driven problems
- Distributionally robust stochastic programming
- Efficient Estimation and Inferences for Varying-Coefficient Models
- Empirical likelihood ratio confidence regions
- Entropic value-at-risk: a new coherent risk measure
- Epi‐consistency of convex stochastic programs
- Finite mixture models
- Higher moment coherent risk measures
- Improving predictive inference under covariate shift by weighting the log-likelihood function
- Inference in hidden Markov models.
- Information-Theoretic Lower Bounds on the Oracle Complexity of Stochastic Convex Optimization
- Lectures on Stochastic Programming
- Linear Minimax Regret Estimation of Deterministic Parameters with Bounded Data Uncertainties
- Local Rademacher complexities
- Maximin effects in inhomogeneous large-scale data
- Minimax optimal control of stochastic uncertain systems with relative entropy constraints
- On Kusuoka representation of law invariant risk measures
- Path-space information bounds for uncertainty quantification and sensitivity analysis of stochastic dynamics
- Quantifying distributional model risk via optimal transport
- Regularization and Variable Selection Via the Elastic Net
- Robust Statistics
- Robust Statistics
- Robust Wasserstein profile inference and applications to machine learning
- Robust analysis in stochastic simulation: computation and performance guarantees
- Robust bounds on risk-sensitive functionals via Rényi divergence
- Robust empirical optimization is almost the same as mean-variance optimization
- Robust optimization
- Robust risk measurement and model risk
- Robust sensitivity analysis for stochastic systems
- Robustness
- Rényi Divergence and Kullback-Leibler Divergence
- Statistical decision functions which minimize the maximum risk
- Statistical estimation in varying coefficient models
- Stochastic orders
- The Brunn-Minkowski inequality
- The Integral of a Symmetric Unimodal Function over a Symmetric Convex Set and Some Probability Inequalities
- The empirical likelihood approach to quantifying uncertainty in sample average approximation
- Variance-based regularization with convex objectives
- Variational Analysis
- Weak convergence and empirical processes. With applications to statistics
Cited in
(39)- Weighted support vector machine for extremely imbalanced data
- scientific article; zbMATH DE number 7626776 (Why is no real title available?)
- Distributionally robust optimization
- Distributionally robust optimization with moment ambiguity sets
- Toward theoretical understandings of robust Markov decision processes: sample complexity and asymptotics
- Distributionally robust chance-constrained kernel-based support vector machine
- Regularized distributionally robust optimization with application to the index tracking problem
- Statistical inference for generative adversarial networks and other minimax problems
- Robust inference for moment condition models without rational expectations
- scientific article; zbMATH DE number 7370573 (Why is no real title available?)
- Risk-adaptive approaches to stochastic optimization: a survey
- Sample complexity of variance-reduced distributionally robust Q-learning
- An introduction to deep survival analysis models for predicting time-to-event outcomes
- The distributionally robust prediction error of the \(\sqrt{\mathrm{LASSO}}\) and related estimators
- Causality-Oriented Robustness: Exploiting General Noise Interventions
- Distributionally robust optimal uncertainty quantification under Phi-divergence ambiguity
- Convexification of fractional distributionally robust probability maximization problems under phi-divergence ambiguity
- Distributionally Robust Losses for Latent Covariate Mixtures
- A stochastic subgradient method for distributionally robust non-convex and non-smooth learning
- Distributionally robust optimization with generalized total variation ambiguity sets
- Robust grouped variable selection using distributionally robust optimization
- Outlier-robust nonsmooth stochastic optimization
- Mean-semideviation-based distributionally robust learning with weakly convex losses: convergence rates and finite-sample guarantees
- Continuous treatment effect estimation via generative adversarial de-confounding
- Superquantiles at work: machine learning applications and efficient subgradient computation
- Optimal sampling of parametric families: implications for machine learning
- Causality pursuit from heterogeneous environments via neural adversarial invariance learning
- A robust learning approach for regression models based on distributionally robust optimization
- Policy Learning with Asymmetric Counterfactual Utilities
- Robust Validation: Confident Predictions Even When Distributions Shift
- Distributionally robust optimization. A review on theory and applications
- Distributionally robust optimization and robust statistics
- Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessment
- Towards out of distribution generalization for problems in mechanics
- Invariant probabilistic prediction
- A survey of learning criteria going beyond the usual risk
- Counterfactual sensitivity and robustness
- Nonlinear distributionally robust optimization
- Robust and distributionally robust optimization models for linear support vector machine
This page was built for publication: Learning models with uniform performance via distributionally robust optimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q820804)
