A smoothing stochastic gradient method for composite optimization
From MaRDI portal
Abstract: We consider the unconstrained optimization problem whose objective function is composed of a smooth and a non-smooth conponents where the smooth component is the expectation a random function. This type of problem arises in some interesting applications in machine learning. We propose a stochastic gradient descent algorithm for this class of optimization problem. When the non-smooth component has a particular structure, we propose another stochastic gradient descent algorithm by incorporating a smoothing method into our first algorithm. The proofs of the convergence rates of these two algorithms are given and we show the numerical performance of our algorithm by applying them to regularized linear regression problems with different sets of synthetic data.
Recommendations
- A stochastic Nesterov's smoothing accelerated method for general nonsmooth constrained stochastic composite convex optimization
- Gradient sliding for composite optimization
- Gradient methods for minimizing composite functions
- Randomized smoothing for stochastic optimization
- An optimal method for stochastic composite optimization
Cites work
- A coordinate gradient descent method for nonsmooth separable minimization
- An optimal method for stochastic composite optimization
- Applications of second-order cone programming
- Gradient methods for minimizing composite functions
- Interior Gradient and Proximal Methods for Convex and Conic Optimization
- Introductory lectures on convex optimization. A basic course.
- Model Selection and Estimation in Regression with Grouped Variables
- Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization I: A Generic Algorithmic Framework
- Primal-dual first-order methods with \({\mathcal {O}(1/\varepsilon)}\) iteration-complexity for cone programming
- Robust Stochastic Approximation Approach to Stochastic Programming
- Smooth minimization of non-smooth functions
- Smoothing proximal gradient method for general structured sparse regression
- Sparsity and Smoothness Via the Fused Lasso
- Structured variable selection with sparsity-inducing norms
- The Group Lasso for Logistic Regression
- Trace norm regularization: reformulations, algorithms, and multi-task learning
Cited in
(14)- Combining search directions using gradient flows
- Conditional gradient type methods for composite nonlinear and stochastic optimization
- Randomized smoothing variance reduction method for large-scale non-smooth convex optimization
- A stochastic Nesterov's smoothing accelerated method for general nonsmooth constrained stochastic composite convex optimization
- Laplacian smoothing gradient descent
- Randomized smoothing for stochastic optimization
- Gradient sliding for composite optimization
- An optimal method for stochastic composite optimization
- A data efficient and feasible level set method for stochastic convex optimization with expectation constraints
- Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions
- Riemannian smoothing gradient type algorithms for nonsmooth optimization problem on compact Riemannian submanifold embedded in Euclidean space
- Unified analysis of stochastic gradient methods for composite convex and smooth optimization
- Stochastic composition optimization of functions without Lipschitz continuous gradient
- Optimal Algorithms for Stochastic Complementary Composite Minimization
This page was built for publication: A smoothing stochastic gradient method for composite optimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2926083)
