Randomized smoothing for stochastic optimization
From MaRDI portal
Abstract: We analyze convergence rates of stochastic optimization procedures for non-smooth convex optimization problems. By combining randomized smoothing techniques with accelerated gradient methods, we obtain convergence rates of stochastic optimization procedures, both in expectation and with high probability, that have optimal dependence on the variance of the gradient estimates. To the best of our knowledge, these are the first variance-based rates for non-smooth optimization. We give several applications of our results to statistical estimation problems, and provide experimental results that demonstrate the effectiveness of the proposed algorithms. We also describe how a combination of our algorithm with recent work on decentralized optimization yields a distributed stochastic optimization algorithm that is order-optimal.
Recommendations
- Randomized smoothing variance reduction method for large-scale non-smooth convex optimization
- A stochastic Nesterov's smoothing accelerated method for general nonsmooth constrained stochastic composite convex optimization
- A smoothing stochastic gradient method for composite optimization
- Accelerated randomized stochastic optimization.
- Gradient-free proximal methods with inexact oracle for convex stochastic nonsmooth optimization problems on the simplex
Cited in
(45)- Leveraging randomized smoothing for optimal control of nonsmooth dynamical systems
- Algorithms for stochastic optimization with function or expectation constraints
- A Stochastic Smoothing Algorithm for Semidefinite Programming
- An inexact variable metric proximal point algorithm for generic quasi-Newton acceleration
- The impact of randomization in smoothing networks
- Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization
- Randomized smoothing variance reduction method for large-scale non-smooth convex optimization
- A new computational framework for log-concave density estimation
- Zero order algorithm for decentralized optimization problems
- A sparsity preserving stochastic gradient methods for sparse regression
- Gradient-free federated learning methods with l₁ and l₂-randomization for non-smooth convex stochastic optimization problems
- Computationally efficient and statistically optimal robust high-dimensional linear regression
- Zeroth-order random subspace algorithm for non-smooth convex optimization
- Optimality and duality for nonconvex fuzzy optimization using granular differentiability method
- Gradient-free method for nonsmooth distributed optimization
- A stochastic Nesterov's smoothing accelerated method for general nonsmooth constrained stochastic composite convex optimization
- A unified mini-batch stochastic accelerated method for nonconvex stochastic programming
- Supervised nonnegative matrix factorization via minimization of regularized Moreau-envelope of divergence function with application to music transcription
- Short paper -- A note on the Frank-Wolfe algorithm for a class of nonconvex and nonsmooth optimization problems
- Optimization on a finer scale: bounded local subgradient variation perspective
- Stochastic mirror descent method for distributed multi-agent optimization
- Penalty methods with stochastic approximation for stochastic nonlinear programming
- Rejoinder
- Algorithms with gradient clipping for stochastic optimization with heavy-tailed noise
- Inexact zeroth-order nonsmooth and nonconvex stochastic composite optimization and applications
- Nonsmooth projection-free optimization with functional constraints
- Stochastic zeroth order descent with structured directions
- A dual approach for optimal algorithms in distributed optimization over networks
- On smoothing, regularization, and averaging in stochastic approximation methods for stochastic variational inequality problems
- On the convergence rate issues of general Markov search for global minimum
- Truncated Cauchy random perturbations for smoothed functional-based stochastic optimization
- scientific article; zbMATH DE number 1748526 (Why is no real title available?)
- Extragradient Method with Variance Reduction for Stochastic Variational Inequalities
- An Improved Unconstrained Approach for Bilevel Optimization
- Optimal convergence rates for convex distributed optimization in networks
- Sample average approximations of strongly convex stochastic programs in Hilbert spaces
- Bias reduction in sample-based optimization
- Incremental gradient-free method for nonsmooth distributed optimization
- Complexity guarantees for an implicit smoothing-enabled method for stochastic MPECs
- A smoothing stochastic gradient method for composite optimization
- Smoothed functional algorithms for stochastic optimization using q-Gaussian distributions
- Lower bounds for parallel and randomized convex optimization
- Convergence properties of stochastic proximal subgradient method in solving a class of composite optimization problems with cardinality regularizer
- Central limit theorems for vector-valued composite functionals with smoothing and applications
- Laplacian smoothing gradient descent
This page was built for publication: Randomized smoothing for stochastic optimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2910890)
