A stochastic subgradient method for distributionally robust non-convex and non-smooth learning
From MaRDI portal
(Redirected from Publication:2159458)
Abstract: We consider a distributionally robust formulation of stochastic optimization problems arising in statistical learning, where robustness is with respect to uncertainty in the underlying data distribution. Our formulation builds on risk-averse optimization techniques and the theory of coherent risk measures. It uses semi-deviation risk for quantifying uncertainty, allowing us to compute solutions that are robust against perturbations in the population data distribution. We consider a large family of loss functions that can be non-convex and non-smooth and develop an efficient stochastic subgradient method. We prove that it converges to a point satisfying the optimality conditions. To our knowledge, this is the first method with rigorous convergence guarantees in the context of non-convex non-smooth distributionally robust stochastic optimization. Our method can achieve any desired level of robustness with little extra computational cost compared to population risk minimization. We also illustrate the performance of our algorithm on real datasets arising in convex and non-convex supervised learning problems.
Recommendations
- Stochastic subgradient descent method for large-scale robust chance-constrained support vector machines
- Stochastic perturbation of subgradient algorithm for nonconvex deep neural networks
- Robust stochastic optimization with convex risk measures: a discretized subgradient scheme
- Stochastic generalized gradient method for nonconvex nonsmooth stochastic optimization
- Stochastic subgradient method for quasi-convex optimization problems
- Inexact stochastic subgradient projection method for stochastic equilibrium problems with nonmonotone bifunctions: application to expected risk minimization in machine learning
- Proximally guided stochastic subgradient method for nonsmooth, nonconvex problems
- A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization
- A stochastic alternating direction method of multipliers for non-smooth and non-convex optimization
- Stochastic conditional gradient methods: from convex minimization to submodular maximization
Cites work
- scientific article; zbMATH DE number 6378127 (Why is no real title available?)
- scientific article; zbMATH DE number 4035561 (Why is no real title available?)
- scientific article; zbMATH DE number 3761782 (Why is no real title available?)
- scientific article; zbMATH DE number 1972910 (Why is no real title available?)
- scientific article; zbMATH DE number 1502618 (Why is no real title available?)
- scientific article; zbMATH DE number 3437904 (Why is no real title available?)
- scientific article; zbMATH DE number 6438182 (Why is no real title available?)
- A Linearization Method for Nonsmooth Stochastic Programming Problems
- A Single Timescale Stochastic Approximation Method for Nested Stochastic Optimization
- A Stochastic Approximation Method
- A Stochastic Subgradient Method for Nonsmooth Nonconvex Multilevel Composition Optimization
- A robust approach based on conditional value-at-risk measure to statistical learning problems
- Accelerating Stochastic Composition Optimization
- Bundle methods for regularized risk minimization
- Coherent measures of risk
- Computationally tractable counterparts of distributionally robust constraints on risk measures
- Convergence of a stochastic subgradient method with averaging for nonsmooth nonconvex constrained optimization
- Convex optimization: algorithms and complexity
- Curves of descent
- Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations
- Deep learning
- From stochastic dominance to mean-risk models: Semideviations as risk measures
- Generalized Gradients and Applications
- Learning models with uniform performance via distributionally robust optimization
- Lectures on Stochastic Programming
- Models and algorithms for distributionally robust least squares problems
- Multilevel stochastic gradient methods for nested composition optimization
- On consistency of stochastic dominance and mean-semideviation models
- Optimal stochastic approximation algorithms for strongly convex stochastic composite optimization. II: Shrinking procedures and optimal algorithms
- Optimization of Convex Risk Functions
- Parallelizing stochastic gradient descent for least squares regression: mini-batching, averaging, and model misspecification
- Sample average approximation method for compound stochastic optimization problems
- Semismooth and Semiconvex Functions in Constrained Optimization
- Statistical estimation of composite risk functionals and risk optimization problems
- Stochastic Methods for Composite and Weakly Convex Optimization Problems
- Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions
- Stochastic finance. An introduction in discrete time
- Stochastic model-based minimization of weakly convex functions
- The landscape of empirical risk for nonconvex losses
- Understanding machine learning. From theory to algorithms
Cited in
(4)- Learning models with uniform performance via distributionally robust optimization
- Distributionally robust optimization with moment ambiguity sets
- Mini-Batch Risk Forms
- Inexact stochastic subgradient projection method for stochastic equilibrium problems with nonmonotone bifunctions: application to expected risk minimization in machine learning
This page was built for publication: A stochastic subgradient method for distributionally robust non-convex and non-smooth learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2159458)
