Stochastic dual coordinate ascent methods for regularized loss minimization
From MaRDI portal
Abstract: Stochastic Gradient Descent (SGD) has become popular for solving large scale supervised machine learning optimization problems such as SVM, due to their strong theoretical guarantees. While the closely related Dual Coordinate Ascent (DCA) method has been implemented in various software packages, it has so far lacked good convergence analysis. This paper presents a new analysis of Stochastic Dual Coordinate Ascent (SDCA) showing that this class of methods enjoy strong theoretical guarantees that are comparable or better than SGD. This analysis justifies the effectiveness of SDCA for practical applications.
Recommendations
- Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization
- A stochastic algorithm with optimal convergence rate for strongly convex optimization problems
- Dual averaging methods for regularized stochastic learning and online optimization
- Optimization methods for large-scale machine learning
- Large-scale machine learning with stochastic gradient descent
Cited in
(only showing first 100 items - show all)- A flexible coordinate descent method
- The complexity of primal-dual fixed point methods for ridge regression
- Linear convergence rate for the MDM algorithm for the nearest point problem
- Inexact proximal stochastic gradient method for convex composite optimization
- Dual block-coordinate forward-backward algorithm with application to deconvolution and deinterlacing of video sequences
- Extended ADMM and BCD for nonseparable convex minimization models with quadratic coupling terms: convergence analysis and insights
- An optimal randomized incremental gradient method
- Parallel decomposition methods for linearly constrained problems subject to simple bound with application to the SVMs training
- Stochastic gradient method with Barzilai-Borwein step for unconstrained nonlinear optimization
- Convergence of stochastic proximal gradient algorithm
- Point process estimation with Mirror Prox algorithms
- Generalized stochastic Frank-Wolfe algorithm with stochastic ``substitute gradient for structured convex optimization
- Analysis of biased stochastic gradient descent using sequential semidefinite programs
- Momentum and stochastic momentum for stochastic gradient, Newton, proximal point and subspace descent methods
- Randomized smoothing variance reduction method for large-scale non-smooth convex optimization
- Stochastic quasi-gradient methods: variance reduction via Jacobian sketching
- A stochastic subspace approach to gradient-free optimization in high dimensions
- Fastest rates for stochastic mirror descent methods
- Inverse optimization approach to the identification of electricity consumer models
- Stochastic DCA for minimizing a large sum of DC functions with application to multi-class logistic regression
- A hybrid stochastic optimization framework for composite nonconvex optimization
- Accelerating mini-batch SARAH by step size rules
- Block-coordinate and incremental aggregated proximal gradient methods for nonsmooth nonconvex problems
- A stochastic extra-step quasi-Newton method for nonsmooth nonconvex optimization
- Finite-sum smooth optimization with SARAH
- Communication-efficient distributed multi-task learning with matrix sparsity regularization
- Linear convergence of cyclic SAGA
- Principal component projection with low-degree polynomials
- Worst-case complexity of cyclic coordinate descent: O(n^2) gap with randomized version
- Near-optimal discrete optimization for experimental design: a regret minimization approach
- Forward-reflected-backward method with variance reduction
- Markov chain block coordinate descent
- Provable accelerated gradient method for nonconvex low rank optimization
- Efficient learning with robust gradient descent
- Dual coordinate ascent methods for non-strictly convex minimization
- Proximal average approximated incremental gradient descent for composite penalty regularized empirical risk minimization
- Generalized forward-backward splitting with penalization for monotone inclusion problems
- An accelerated variance reducing stochastic method with Douglas-Rachford splitting
- Negotiating multicollinearity with spike-and-slab priors
- Concentration inequalities for sampling without replacement
- Improving kernel online learning with a snapshot memory
- Cocoercivity, smoothness and bias in variance-reduced stochastic gradient methods
- Variance reduction for root-finding problems
- Accelerated stochastic variance reduction for a class of convex optimization problems
- Optimization in high dimensions via accelerated, parallel, and proximal coordinate descent
- Adaptive sampling for incremental optimization using stochastic gradient descent
- On data preconditioning for regularized loss minimization
- Stochastic block mirror descent methods for nonsmooth and stochastic optimization
- Inexact coordinate descent: complexity and preconditioning
- On optimal probabilities in stochastic coordinate descent methods
- A randomized coordinate descent method with volume sampling
- A Randomized Exchange Algorithm for Computing Optimal Approximate Designs of Experiments
- Differentially Private Distributed Learning
- Accelerated, parallel, and proximal coordinate descent
- The cyclic block conditional gradient method for convex optimization problems
- An accelerated randomized proximal coordinate gradient method and its application to regularized empirical risk minimization
- Distributed block coordinate descent for minimizing partially separable functions
- scientific article; zbMATH DE number 6982318 (Why is no real title available?)
- Nonasymptotic convergence of stochastic proximal point methods for constrained convex optimization
- scientific article; zbMATH DE number 6982970 (Why is no real title available?)
- Katyusha: the first direct acceleration of stochastic gradient methods
- Parallelizing stochastic gradient descent for least squares regression: mini-batching, averaging, and model misspecification
- scientific article; zbMATH DE number 6982986 (Why is no real title available?)
- A new filter‐based stochastic gradient algorithm for dual‐rate ARX models
- A stochastic variance reduction method for PCA by an exact penalty approach
- Optimizing adaptive importance sampling by stochastic approximation
- A randomized nonmonotone block proximal gradient method for a class of structured nonlinear programming
- Avoiding Communication in Primal and Dual Block Coordinate Descent Methods
- Improved asynchronous parallel optimization analysis for stochastic incremental methods
- Utilizing second order information in minibatch stochastic variance reduced proximal iterations
- Stochastic primal-dual coordinate method for regularized empirical risk minimization
- Approximation vector machines for large-scale online learning
- A general distributed dual coordinate optimization framework for regularized loss minimization
- Second-order stochastic optimization for machine learning in linear time
- Distributed stochastic variance reduced gradient methods by sampling extra data with replacement
- Linear coupling: an ultimate unification of gradient and mirror descent
- On the complexity of parallel coordinate descent
- Surpassing gradient descent provably: a cyclic incremental method with linear convergence rate
- Optimization methods for large-scale machine learning
- A coordinate-descent primal-dual algorithm with large step size and possibly nonseparable functions
- Random gradient extrapolation for distributed and stochastic optimization
- Efficient random coordinate descent algorithms for large-scale structured nonconvex optimization
- On the complexity analysis of the primal solutions for the accelerated randomized dual coordinate ascent
- Stochastic nested variance reduction for nonconvex optimization
- scientific article; zbMATH DE number 7255141 (Why is no real title available?)
- L-SVRG and L-Katyusha with arbitrary sampling
- A stochastic alternating direction method of multipliers for non-smooth and non-convex optimization
- Proximal Gradient Methods for Machine Learning and Imaging
- scientific article; zbMATH DE number 7625177 (Why is no real title available?)
- Tensor Canonical Correlation Analysis With Convergence and Statistical Guarantees
- Adaptivity of stochastic gradient methods for nonconvex optimization
- A new homotopy proximal variable-metric framework for composite convex minimization
- On the convergence of stochastic primal-dual hybrid gradient
- A novel Frank-Wolfe algorithm. Analysis and applications to large-scale SVM training
- Sketched Newton-Raphson
- On the efficiency of random permutation for ADMM and coordinate descent
- Stochastic reformulations of linear systems: algorithms and convergence theory
- Analyzing random permutations for cyclic coordinate descent
- Dual iterative hard thresholding
- A method with convergence rates for optimization problems with variational inequality constraints
This page was built for publication: Stochastic dual coordinate ascent methods for regularized loss minimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5405257)
