Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions

DOI10.1007/s10107-016-1017-3zbMath1356.90099arXiv1411.3803OpenAlexW1494085563MaRDI QIDQ507334

Mengdi Wang, Ethan X. Fang, Han Liu

Publication date: 3 February 2017

Published in: Mathematical Programming. Series A. Series B (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1411.3803

zbMATH Keywords

stochastic optimizationconvex optimizationsimulationstatistical learningstochastic gradientsample complexity

Mathematics Subject Classification ID

Convex programming (90C25) Large-scale problems in mathematical programming (90C06) Stochastic programming (90C15) Online algorithms; streaming algorithms (68W27)

Related Items

A Two-Timescale Stochastic Algorithm Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Critic, Adaptive primal-dual stochastic gradient method for expectation-constrained convex stochastic programs, Zeroth-Order Stochastic Compositional Algorithms for Risk-Aware Learning, A stochastic subgradient method for distributionally robust non-convex and non-smooth learning, Stochastic Multilevel Composition Optimization Algorithms with Level-Independent Convergence Rates, Streaming constrained binary logistic regression with online standardized data, Stochastic Methods for Composite and Weakly Convex Optimization Problems, A convergence analysis of the perturbed compositional gradient flow: averaging principle and normal deviations, On the information-adaptive variants of the ADMM: an iteration complexity perspective, Probability maximization via Minkowski functionals: convex representations and tractable resolution, Hybrid SGD algorithms to solve stochastic composite optimization problems with application in sparse portfolio selection problems, Stochastic composition optimization of functions without Lipschitz continuous gradient, Unnamed Item, On the sample complexity of actor-critic method for reinforcement learning with function approximation, A Stochastic Composite Augmented Lagrangian Method for Reinforcement Learning, Distributed stochastic compositional optimization problems over directed networks, Primal-Dual Stochastic Gradient Method for Convex Programs with Many Functional Constraints, A single timescale stochastic quasi-Newton method for stochastic optimization, A Two-Time-Scale Stochastic Optimization Framework with Applications in Control and Reinforcement Learning, Sample Complexity of Sample Average Approximation for Conditional Stochastic Optimization, Stochastic Model-Based Minimization of Weakly Convex Functions, Multilevel Stochastic Gradient Methods for Nested Composition Optimization, Near-optimal stochastic approximation for online principal component estimation, Dynamic stochastic approximation for multi-stage stochastic optimization, A Single Timescale Stochastic Approximation Method for Nested Stochastic Optimization, Proximally Guided Stochastic Subgradient Method for Nonsmooth, Nonconvex Problems, MultiLevel Composite Stochastic Optimization via Nested Variance Reduction, Accelerating Stochastic Composition Optimization, A Stochastic Subgradient Method for Nonsmooth Nonconvex Multilevel Composition Optimization, Stochastic variance-reduced prox-linear algorithms for nonconvex composite optimization, Finite-Time Analysis and Restarting Scheme for Linear Two-Time-Scale Stochastic Approximation, Unnamed Item, Solving Nonsmooth and Nonconvex Compound Stochastic Programs with Applications to Risk Measure Minimization, A hybrid stochastic optimization framework for composite nonconvex optimization

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Random algorithms for convex minimization problems
• Incremental proximal methods for large scale convex optimization
• High-dimensional analysis of semidefinite relaxations for sparse principal components
• Variable selection in nonparametric additive models
• Stochastic approximation. A dynamical systems viewpoint.
• Stochastic approximation with two time scales
• Convergence rate of linear two-time-scale stochastic approximation.
• Incremental constraint projection methods for variational inequalities
• Statistical estimation of composite risk functionals and risk optimization problems
• Incremental Subgradient Methods for Nondifferentiable Optimization
• On Upper Functions for Stochastic Approximation Procedures
• Model selection and estimation in the Gaussian graphical model
• Lectures on Stochastic Programming
• Acceleration of Stochastic Approximation by Averaging
• Sparse Additive Models
• Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization I: A Generic Algorithmic Framework
• A two Timescale Stochastic Approximation Scheme for Simulation-Based Parametric Optimization
• Information-Theoretic Lower Bounds on the Oracle Complexity of Stochastic Convex Optimization
• Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization, II: Shrinking Procedures and Optimal Algorithms
• Stochastic Estimation of the Maximum of a Regression Function