Distributed coordinate descent method for learning with big data
From MaRDI portal
Abstract: In this paper we develop and analyze Hydra: HYbriD cooRdinAte descent method for solving loss minimization problems with big data. We initially partition the coordinates (features) and assign each partition to a different node of a cluster. At every iteration, each node picks a random subset of the coordinates from those it owns, independently from the other computers, and in parallel computes and applies updates to the selected coordinates based on a simple closed-form formula. We give bounds on the number of iterations sufficient to approximately solve the problem with high probability, and show how it depends on the data and on the partitioning. We perform numerical experiments with a LASSO instance described by a 3TB matrix.
Recommendations
- Parallel coordinate descent methods for big data optimization
- Efficiency of coordinate descent methods on huge-scale optimization problems
- Coordinate descent algorithms
- Accelerated, parallel, and proximal coordinate descent
- Optimization in high dimensions via accelerated, parallel, and proximal coordinate descent
Cited in
(40)- An accelerated distributed gradient method with local memory
- Parallel coordinate descent methods for big data optimization
- Distributed Learning with Sparse Communications by Identification
- On optimal probabilities in stochastic coordinate descent methods
- Distributed cooperative learning over time-varying random networks using a gossip-based communication protocol
- scientific article; zbMATH DE number 6982318 (Why is no real title available?)
- Convergence in distribution of randomized algorithms: the case of partially separable optimization
- Primal-dual block-proximal splitting for a class of non-convex problems
- A class of parallel doubly stochastic algorithms for large-scale learning
- A generic coordinate descent solver for non-smooth convex optimisation
- scientific article; zbMATH DE number 7306895 (Why is no real title available?)
- On Adaptive Sketch-and-Project for Solving Linear Systems
- A distributed block coordinate descent method for training l₁ regularized linear classifiers
- scientific article; zbMATH DE number 7733450 (Why is no real title available?)
- An accelerated randomized proximal coordinate gradient method and its application to regularized empirical risk minimization
- Matrix completion under interval uncertainty
- An efficient distributed learning algorithm based on effective local functional approximations
- Distributed Sufficient Dimension Reduction for Heterogeneous Massive Data
- Stochastic reformulations of linear systems: algorithms and convergence theory
- Efficiency of the accelerated coordinate descent method on structured optimization problems
- Distributed statistical optimization for non-randomly stored big data with application to penalized learning
- Accelerated, parallel, and proximal coordinate descent
- Parallel random coordinate descent method for composite minimization: convergence analysis and error bounds
- Parallel random block-coordinate forward-backward algorithm: a unified convergence analysis
- Block-proximal methods with spatially adapted acceleration
- Distributed multi-agent optimisation via coordination with second-order nearest neighbours
- Additive-effect assisted learning
- Stochastic distributed learning with gradient quantization and double-variance reduction
- A Randomized Exchange Algorithm for Computing Optimal Approximate Designs of Experiments
- Optimization in high dimensions via accelerated, parallel, and proximal coordinate descent
- Accelerated methods with compression for horizontal and vertical federated learning
- Parallel block coordinate descent methods with identification strategies
- On the convergence analysis of asynchronous SGD for solving consistent linear systems
- Manifold optimization for hybrid beamforming in dual-function radar-communication system
- Distributed learning with compressed gradient differences
- scientific article; zbMATH DE number 6982986 (Why is no real title available?)
- On the complexity of parallel coordinate descent
- Distributed block coordinate descent for minimizing partially separable functions
- More communication-efficient distributed sparse learning
- DSCOVR: randomized primal-dual block coordinate algorithms for asynchronous distributed optimization
This page was built for publication: Distributed coordinate descent method for learning with big data
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2810888)
