Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization I: A Generic Algorithmic Framework
From MaRDI portal
Publication:4915174
DOI10.1137/110848864zbMath1301.62077OpenAlexW2045744861MaRDI QIDQ4915174
Publication date: 9 April 2013
Published in: SIAM Journal on Optimization (Search for Journal in Brave)
Full work available at URL: https://semanticscholar.org/paper/1ed037050fe938c277915145a941292eac6f7a58
Convex programming (90C25) Stochastic programming (90C15) Stochastic approximation (62L20) Approximation algorithms (68W25)
Related Items (83)
Lp and almost sure rates of convergence of averaged stochastic gradient algorithms: locally strongly convex objective ⋮ On inexact stochastic splitting methods for a class of nonconvex composite optimization problems with relative error ⋮ Stochastic forward-backward splitting for monotone inclusions ⋮ On Hermite-Hadamard type inequalities for \(n \)-polynomial convex stochastic processes ⋮ Stochastic approximation method using diagonal positive-definite matrices for convex optimization with fixed point constraints ⋮ Stochastic intermediate gradient method for convex optimization problems ⋮ Accelerated gradient sliding for structured convex optimization ⋮ Gradient sliding for composite optimization ⋮ New results on subgradient methods for strongly convex optimization problems with a unified analysis ⋮ A smoothing stochastic gradient method for composite optimization ⋮ Proximal average approximated incremental gradient descent for composite penalty regularized empirical risk minimization ⋮ Unnamed Item ⋮ Unnamed Item ⋮ Simple and Optimal Methods for Stochastic Variational Inequalities, II: Markovian Noise and Policy Evaluation in Reinforcement Learning ⋮ On the information-adaptive variants of the ADMM: an iteration complexity perspective ⋮ Accelerated Stochastic Algorithms for Convex-Concave Saddle-Point Problems ⋮ Statistical inference for model parameters in stochastic gradient descent ⋮ Algorithms for stochastic optimization with function or expectation constraints ⋮ Accelerated schemes for a class of variational inequalities ⋮ On smoothing, regularization, and averaging in stochastic approximation methods for stochastic variational inequality problems ⋮ Distributed communication-sliding mirror-descent algorithm for nonsmooth resource allocation problem ⋮ A dual-based stochastic inexact algorithm for a class of stochastic nonsmooth convex composite problems ⋮ Non-smooth setting of stochastic decentralized convex optimization problem over time-varying graphs ⋮ Universal Conditional Gradient Sliding for Convex Optimization ⋮ Stochastic Block Mirror Descent Methods for Nonsmooth and Stochastic Optimization ⋮ An overview of stochastic quasi-Newton methods for large-scale machine learning ⋮ Graph Topology Invariant Gradient and Sampling Complexity for Decentralized and Stochastic Optimization ⋮ Unnamed Item ⋮ Inexact proximal stochastic gradient method for convex composite optimization ⋮ Optimal Algorithms for Stochastic Complementary Composite Minimization ⋮ Theoretical analysis of Adam using hyperparameters close to one without Lipschitz smoothness ⋮ On the rates of convergence of parallelized averaged stochastic gradient algorithms ⋮ Nonlinear Gradient Mappings and Stochastic Optimization: A General Framework with Applications to Heavy-Tail Noise ⋮ Block mirror stochastic gradient method for stochastic optimization ⋮ First-order methods for convex optimization ⋮ Incremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learning ⋮ Differentially private inference via noisy optimization ⋮ Reliable Error Estimates for Optimal Control of Linear Elliptic PDEs with Random Inputs ⋮ On optimal universal first-order methods for minimizing heterogeneous sums ⋮ Bregman proximal point type algorithms for quasiconvex minimization ⋮ Adaptive proximal SGD based on new estimating sequences for sparser ERM ⋮ Algorithms with gradient clipping for stochastic optimization with heavy-tailed noise ⋮ Penalty methods with stochastic approximation for stochastic nonlinear programming ⋮ Inexact SA method for constrained stochastic convex SDP and application in Chinese stock market ⋮ A sparsity preserving stochastic gradient methods for sparse regression ⋮ Two stochastic optimization algorithms for convex optimization with fixed point constraints ⋮ Recent theoretical advances in decentralized distributed convex optimization ⋮ Portfolio selection with the effect of systematic risk diversification: formulation and accelerated gradient algorithm ⋮ On variance reduction for stochastic smooth convex optimization with multiplicative noise ⋮ Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions ⋮ Multistep stochastic mirror descent for risk-averse convex stochastic programs based on extended polyhedral risk measures ⋮ Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization ⋮ Convergence of stochastic proximal gradient algorithm ⋮ Dynamic stochastic approximation for multi-stage stochastic optimization ⋮ An optimal randomized incremental gradient method ⋮ Random Gradient Extrapolation for Distributed and Stochastic Optimization ⋮ Unnamed Item ⋮ Optimizing cluster structures with inner product induced norm based dissimilarity measures: theoretical development and convergence analysis ⋮ Optimal stochastic extragradient schemes for pseudomonotone stochastic variational inequality problems and their variants ⋮ Stochastic intermediate gradient method for convex problems with stochastic inexact oracle ⋮ Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization ⋮ Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization ⋮ Iteration-complexity of first-order augmented Lagrangian methods for convex programming ⋮ Algorithms of robust stochastic optimization based on mirror descent method ⋮ Bundle-level type methods uniformly optimal for smooth and nonsmooth convex optimization ⋮ Fast and safe: accelerated gradient methods with optimality certificates and underestimate sequences ⋮ Fastest rates for stochastic mirror descent methods ⋮ Inexact stochastic subgradient projection method for stochastic equilibrium problems with nonmonotone bifunctions: application to expected risk minimization in machine learning ⋮ Communication-efficient algorithms for decentralized and stochastic optimization ⋮ Robust extremum seeking for a second order uncertain plant using a sliding mode controller ⋮ Conditional Gradient Sliding for Convex Optimization ⋮ A family of subgradient-based methods for convex optimization problems in a unifying framework ⋮ Stochastic (Approximate) Proximal Point Methods: Convergence, Optimality, and Adaptivity ⋮ On the Solution of Stochastic Optimization and Variational Problems in Imperfect Information Regimes ⋮ Computing the Best Approximation over the Intersection of a Polyhedral Set and the Doubly Nonnegative Cone ⋮ Unnamed Item ⋮ Unnamed Item ⋮ Convergence analysis of the stochastic reflected forward-backward splitting algorithm ⋮ Robust Accelerated Gradient Methods for Smooth Strongly Convex Functions ⋮ Smoothed Variable Sample-Size Accelerated Proximal Methods for Nonsmooth Stochastic Convex Programs ⋮ On stochastic accelerated gradient with convergence rate ⋮ On strongly quasiconvex functions: existence results and proximal point algorithms ⋮ Accelerated gradient methods for nonconvex nonlinear and stochastic programming
Uses Software
This page was built for publication: Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization I: A Generic Algorithmic Framework
