Stochastic dual coordinate ascent methods for regularized loss minimization
From MaRDI portal
Abstract: Stochastic Gradient Descent (SGD) has become popular for solving large scale supervised machine learning optimization problems such as SVM, due to their strong theoretical guarantees. While the closely related Dual Coordinate Ascent (DCA) method has been implemented in various software packages, it has so far lacked good convergence analysis. This paper presents a new analysis of Stochastic Dual Coordinate Ascent (SDCA) showing that this class of methods enjoy strong theoretical guarantees that are comparable or better than SGD. This analysis justifies the effectiveness of SDCA for practical applications.
Recommendations
- Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization
- A stochastic algorithm with optimal convergence rate for strongly convex optimization problems
- Dual averaging methods for regularized stochastic learning and online optimization
- Optimization methods for large-scale machine learning
- Large-scale machine learning with stochastic gradient descent
Cited in
(only showing first 100 items - show all)- Distributed block-diagonal approximation methods for regularized empirical risk minimization
- High-dimensional model recovery from random sketched data by exploring intrinsic sparsity
- Generalized forward-backward splitting with penalization for monotone inclusion problems
- Learning with risks based on M-location
- SVRG meets AdaGrad: painless variance reduction
- Differentially Private Distributed Learning
- Communication-efficient distributed multi-task learning with matrix sparsity regularization
- SPIRAL: a superlinearly convergent incremental proximal algorithm for nonconvex finite sum minimization
- Variance reduction for root-finding problems
- A stochastic algorithm with optimal convergence rate for strongly convex optimization problems
- Stochastic primal-dual coordinate method for regularized empirical risk minimization
- Concentration inequalities for sampling without replacement
- Linear convergence of cyclic SAGA
- Markov chain block coordinate descent
- scientific article; zbMATH DE number 7625177 (Why is no real title available?)
- A stochastic alternating direction method of multipliers for non-smooth and non-convex optimization
- Dual block-coordinate forward-backward algorithm with application to deconvolution and deinterlacing of video sequences
- scientific article; zbMATH DE number 7306858 (Why is no real title available?)
- A stochastic extra-step quasi-Newton method for nonsmooth nonconvex optimization
- Parallelizing stochastic gradient descent for least squares regression: mini-batching, averaging, and model misspecification
- A new filter‐based stochastic gradient algorithm for dual‐rate ARX models
- Worst-case complexity of cyclic coordinate descent: \(O(n^2)\) gap with randomized version
- Stochastic gradient methods with preconditioned updates
- Preserving privacy between features in distributed estimation
- Minimizing finite sums with the stochastic average gradient
- Dual coordinate ascent methods for non-strictly convex minimization
- A stochastic variance reduction method for PCA by an exact penalty approach
- scientific article; zbMATH DE number 6982986 (Why is no real title available?)
- Analysis of biased stochastic gradient descent using sequential semidefinite programs
- A randomized coordinate descent method with volume sampling
- On the complexity of parallel coordinate descent
- Distributed block coordinate descent for minimizing partially separable functions
- Riemannian Stochastic Variance Reduced Gradient Algorithm with Retraction and Vector Transport
- A general distributed dual coordinate optimization framework for regularized loss minimization
- Surpassing gradient descent provably: a cyclic incremental method with linear convergence rate
- Adaptive sampling for incremental optimization using stochastic gradient descent
- Two symmetrized coordinate descent methods can be \(O(n^2)\) times slower than the randomized version
- Convergence properties of a randomized primal-dual algorithm with applications to parallel MRI
- Negotiating multicollinearity with spike-and-slab priors
- Randomized block proximal damped Newton method for composite self-concordant minimization
- Efficient learning with robust gradient descent
- An Optimal Algorithm for Decentralized Finite-Sum Optimization
- Adaptivity of stochastic gradient methods for nonconvex optimization
- Block-coordinate and incremental aggregated proximal gradient methods for nonsmooth nonconvex problems
- Inexact coordinate descent: complexity and preconditioning
- On optimal probabilities in stochastic coordinate descent methods
- scientific article; zbMATH DE number 6253925 (Why is no real title available?)
- scientific article; zbMATH DE number 6982318 (Why is no real title available?)
- An accelerated variance reducing stochastic method with Douglas-Rachford splitting
- Avoiding Communication in Primal and Dual Block Coordinate Descent Methods
- Linear coupling: an ultimate unification of gradient and mirror descent
- Principal component projection with low-degree polynomials
- General procedure to provide high-probability guarantees for stochastic saddle point problems
- Stochastic nested variance reduction for nonconvex optimization
- A generic coordinate descent solver for non-smooth convex optimisation
- Near-optimal discrete optimization for experimental design: a regret minimization approach
- Forward-reflected-backward method with variance reduction
- On data preconditioning for regularized loss minimization
- Extended ADMM and BCD for nonseparable convex minimization models with quadratic coupling terms: convergence analysis and insights
- Approximation vector machines for large-scale online learning
- Second-order stochastic optimization for machine learning in linear time
- Proximal Gradient Methods for Machine Learning and Imaging
- Randomized smoothing variance reduction method for large-scale non-smooth convex optimization
- The cyclic block conditional gradient method for convex optimization problems
- Optimization methods for large-scale machine learning
- Nonasymptotic convergence of stochastic proximal point methods for constrained convex optimization
- An accelerated communication-efficient primal-dual optimization framework for structured machine learning
- A coordinate-descent primal-dual algorithm with large step size and possibly nonseparable functions
- On the efficiency of random permutation for ADMM and coordinate descent
- Fastest rates for stochastic mirror descent methods
- An accelerated randomized proximal coordinate gradient method and its application to regularized empirical risk minimization
- Accelerating mini-batch SARAH by step size rules
- scientific article; zbMATH DE number 7370629 (Why is no real title available?)
- A new homotopy proximal variable-metric framework for composite convex minimization
- Stochastic reformulations of linear systems: algorithms and convergence theory
- Stochastic gradient method with Barzilai-Borwein step for unconstrained nonlinear optimization
- Point process estimation with Mirror Prox algorithms
- Generalized stochastic Frank-Wolfe algorithm with stochastic ``substitute gradient for structured convex optimization
- Momentum and stochastic momentum for stochastic gradient, Newton, proximal point and subspace descent methods
- An optimal randomized incremental gradient method
- Accelerating stochastic sequential quadratic programming for equality constrained optimization using predictive variance reduction
- A flexible coordinate descent method
- Utilizing second order information in minibatch stochastic variance reduced proximal iterations
- Accelerated stochastic variance reduction for a class of convex optimization problems
- Efficient random coordinate descent algorithms for large-scale structured nonconvex optimization
- Accelerated, parallel, and proximal coordinate descent
- Stochastic block mirror descent methods for nonsmooth and stochastic optimization
- Provable accelerated gradient method for nonconvex low rank optimization
- Finite-sum smooth optimization with SARAH
- Stochastic quasi-gradient methods: variance reduction via Jacobian sketching
- Local linear convergence of proximal coordinate descent algorithm
- Sketched Newton-Raphson
- Random gradient extrapolation for distributed and stochastic optimization
- Proximal average approximated incremental gradient descent for composite penalty regularized empirical risk minimization
- On the convergence of stochastic primal-dual hybrid gradient
- Stochastic proximal quasi-Newton methods for non-convex composite optimization
- A stochastic subspace approach to gradient-free optimization in high dimensions
- Inverse optimization approach to the identification of electricity consumer models
- Stochastic distributed learning with gradient quantization and double-variance reduction
- scientific article; zbMATH DE number 7400716 (Why is no real title available?)
This page was built for publication: Stochastic dual coordinate ascent methods for regularized loss minimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5405257)
