A stochastic subgradient method for distributionally robust non-convex and non-smooth learning
From MaRDI portal
Publication:2159458
DOI10.1007/S10957-022-02063-6zbMATH Open1492.90103arXiv2006.04873OpenAlexW4284970065MaRDI QIDQ2159458FDOQ2159458
Landi Zhu, Mert Gürbüzbalaban, Andrzej Ruszczyński
Publication date: 1 August 2022
Published in: Journal of Optimization Theory and Applications (Search for Journal in Brave)
Abstract: We consider a distributionally robust formulation of stochastic optimization problems arising in statistical learning, where robustness is with respect to uncertainty in the underlying data distribution. Our formulation builds on risk-averse optimization techniques and the theory of coherent risk measures. It uses semi-deviation risk for quantifying uncertainty, allowing us to compute solutions that are robust against perturbations in the population data distribution. We consider a large family of loss functions that can be non-convex and non-smooth and develop an efficient stochastic subgradient method. We prove that it converges to a point satisfying the optimality conditions. To our knowledge, this is the first method with rigorous convergence guarantees in the context of non-convex non-smooth distributionally robust stochastic optimization. Our method can achieve any desired level of robustness with little extra computational cost compared to population risk minimization. We also illustrate the performance of our algorithm on real datasets arising in convex and non-convex supervised learning problems.
Full work available at URL: https://arxiv.org/abs/2006.04873
Recommendations
- Stochastic subgradient descent method for large-scale robust chance-constrained support vector machines
- Stochastic perturbation of subgradient algorithm for nonconvex deep neural networks
- Robust stochastic optimization with convex risk measures: a discretized subgradient scheme
- Stochastic generalized gradient method for nonconvex nonsmooth stochastic optimization
- Stochastic subgradient method for quasi-convex optimization problems
- Inexact stochastic subgradient projection method for stochastic equilibrium problems with nonmonotone bifunctions: application to expected risk minimization in machine learning
- Proximally guided stochastic subgradient method for nonsmooth, nonconvex problems
- A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization
- A stochastic alternating direction method of multipliers for non-smooth and non-convex optimization
- Stochastic conditional gradient methods: from convex minimization to submodular maximization
risk measuresnon-smooth optimizationrobust learningstochastic subgradient methodcomposition optimization
Cites Work
- Coherent measures of risk
- Title not available (Why is that?)
- Title not available (Why is that?)
- Title not available (Why is that?)
- A Stochastic Approximation Method
- Title not available (Why is that?)
- Understanding Machine Learning
- From stochastic dominance to mean-risk models: Semideviations as risk measures
- Title not available (Why is that?)
- Semismooth and Semiconvex Functions in Constrained Optimization
- Title not available (Why is that?)
- Title not available (Why is that?)
- Stochastic finance. An introduction in discrete time
- On consistency of stochastic dominance and mean-semideviation models
- Lectures on Stochastic Programming
- Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization, II: Shrinking Procedures and Optimal Algorithms
- Bundle methods for regularized risk minimization
- Generalized Gradients and Applications
- Optimization of Convex Risk Functions
- Convex optimization: algorithms and complexity
- Models and algorithms for distributionally robust least squares problems
- Stochastic Methods for Composite and Weakly Convex Optimization Problems
- Title not available (Why is that?)
- Sample Average Approximation Method for Compound Stochastic Optimization Problems
- Statistical estimation of composite risk functionals and risk optimization problems
- Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions
- A robust approach based on conditional value-at-risk measure to statistical learning problems
- A Linearization Method for Nonsmooth Stochastic Programming Problems
- Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations
- Learning models with uniform performance via distributionally robust optimization
- Curves of descent
- Accelerating Stochastic Composition Optimization
- The landscape of empirical risk for nonconvex losses
- Computationally tractable counterparts of distributionally robust constraints on risk measures
- Title not available (Why is that?)
- Stochastic Model-Based Minimization of Weakly Convex Functions
- A Single Timescale Stochastic Approximation Method for Nested Stochastic Optimization
- Convergence of a stochastic subgradient method with averaging for nonsmooth nonconvex constrained optimization
- Multilevel Stochastic Gradient Methods for Nested Composition Optimization
- A Stochastic Subgradient Method for Nonsmooth Nonconvex Multilevel Composition Optimization
Cited In (3)
Uses Software
This page was built for publication: A stochastic subgradient method for distributionally robust non-convex and non-smooth learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2159458)
