Stochastic block mirror descent methods for nonsmooth and stochastic optimization
From MaRDI portal
Abstract: In this paper, we present a new stochastic algorithm, namely the stochastic block mirror descent (SBMD) method for solving large-scale nonsmooth and stochastic optimization problems. The basic idea of this algorithm is to incorporate the block-coordinate decomposition and an incremental block averaging scheme into the classic (stochastic) mirror-descent method, in order to significantly reduce the cost per iteration of the latter algorithm. We establish the rate of convergence of the SBMD method along with its associated large-deviation results for solving general nonsmooth and stochastic optimization problems. We also introduce different variants of this method and establish their rate of convergence for solving strongly convex, smooth, and composite optimization problems, as well as certain nonconvex optimization problems. To the best of our knowledge, all these developments related to the SBMD methods are new in the stochastic optimization literature. Moreover, some of our results also seem to be new for block coordinate descent methods for deterministic optimization.
Recommendations
- Block mirror stochastic gradient method for stochastic optimization
- Gradient-free proximal methods with inexact oracle for convex stochastic nonsmooth optimization problems on the simplex
- Block stochastic gradient iteration for convex and nonconvex optimization
- An optimal method for stochastic composite optimization
- A dual-based stochastic inexact algorithm for a class of stochastic nonsmooth convex composite problems
Cites work
- A coordinate gradient descent method for nonsmooth separable minimization
- A Stochastic Approximation Method
- Acceleration of Stochastic Approximation by Averaging
- An optimal method for stochastic composite optimization
- Bundle-level type methods uniformly optimal for smooth and nonsmooth convex optimization
- Deterministic and stochastic primal-dual subgradient algorithms for uniformly convex minimization
- Efficiency of coordinate descent methods on huge-scale optimization problems
- scientific article; zbMATH DE number 3790208 (Why is no real title available?)
- scientific article; zbMATH DE number 6253925 (Why is no real title available?)
- scientific article; zbMATH DE number 3296905 (Why is no real title available?)
- Introductory lectures on convex optimization. A basic course.
- Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function
- Iteration-complexity of first-order augmented Lagrangian methods for convex programming
- Iteration-complexity of first-order penalty methods for convex programming
- Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization
- Mirror descent and nonlinear projected subgradient methods for convex optimization.
- On stochastic subgradient mirror-descent algorithm with weighted averaging
- On the Convergence of a Matrix Splitting Algorithm for the Symmetric Monotone Linear Complementarity Problem
- On the convergence of block coordinate descent type methods
- Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization I: A Generic Algorithmic Framework
- Optimal stochastic approximation algorithms for strongly convex stochastic composite optimization. II: Shrinking procedures and optimal algorithms
- Randomized methods for linear constraints: convergence rates and conditioning
- Robust Stochastic Approximation Approach to Stochastic Programming
- Smooth minimization of nonsmooth functions with parallel coordinate descent methods
- Stochastic dual coordinate ascent methods for regularized loss minimization
- Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming
- Subgradient methods for huge-scale optimization problems
- Validation analysis of mirror descent stochastic approximation method
Cited in
(38)- Algorithms of inertial mirror descent in stochastic convex optimization problems
- On stochastic mirror-prox algorithms for stochastic Cartesian variational inequalities: randomized block coordinate and optimal averaging schemes
- Conditional gradient type methods for composite nonlinear and stochastic optimization
- Point process estimation with Mirror Prox algorithms
- An accelerated directional derivative method for smooth stochastic convex optimization
- A unified convergence analysis of stochastic Bregman proximal gradient and extragradient methods
- Fastest rates for stochastic mirror descent methods
- On the local convergence of a stochastic semismooth Newton method for nonsmooth nonconvex optimization
- Markov chain block coordinate descent
- Randomized primal-dual proximal block coordinate updates
- On the convergence of asynchronous parallel iteration with unbounded delays
- Accelerated gradient methods for nonconvex nonlinear and stochastic programming
- Distributed constraint-coupled optimization via primal decomposition over random time-varying graphs
- Block stochastic gradient iteration for convex and nonconvex optimization
- Penalty methods with stochastic approximation for stochastic nonlinear programming
- On optimal probabilities in stochastic coordinate descent methods
- Asynchronous variance-reduced block schemes for composite non-convex stochastic optimization: block-specific steplengths and adapted batch-sizes
- A fully stochastic second-order trust region method
- Optimization-based calibration of simulation input models
- A method with convergence rates for optimization problems with variational inequality constraints
- Efficient search of first-order Nash equilibria in nonconvex-concave smooth min-max problems
- Accelerated Stochastic Algorithms for Nonconvex Finite-Sum and Multiblock Optimization
- A stochastic semismooth Newton method for nonsmooth nonconvex optimization
- Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization
- Smoothed Variable Sample-Size Accelerated Proximal Methods for Nonsmooth Stochastic Convex Programs
- Block coordinate type methods for optimization and learning
- Block Policy Mirror Descent
- A stochastic variance reduction algorithm with Bregman distances for structured composite problems
- Stochastic incremental mirror descent algorithms with Nesterov smoothing
- Block mirror stochastic gradient method for stochastic optimization
- Unifying framework for accelerated randomized methods in convex optimization
- Recent Theoretical Advances in Non-Convex Optimization
- An accelerated stochastic mirror descent method
- Bregman distance regularization for nonsmooth and nonconvex optimization
- Randomized block coordinate DC algorithm
- Inertial accelerated stochastic mirror descent for large-scale generalized tensor CP decomposition
- Validation analysis of mirror descent stochastic approximation method
- Randomized subspace correction methods for convex optimization
This page was built for publication: Stochastic block mirror descent methods for nonsmooth and stochastic optimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2954396)
