Decentralized dictionary learning over time-varying digraphs
From MaRDI portal
Abstract: This paper studies Dictionary Learning problems wherein the learning task is distributed over a multi-agent network, modeled as a time-varying directed graph. This formulation is relevant, for instance, in Big Data scenarios where massive amounts of data are collected/stored in different locations (e.g., sensors, clouds) and aggregating and/or processing all data in a fusion center might be inefficient or unfeasible, due to resource limitations, communication overheads or privacy issues. We develop a unified decentralized algorithmic framework for this class of nonconvex problems, which is proved to converge to stationary solutions at a sublinear rate. The new method hinges on Successive Convex Approximation techniques, coupled with a decentralized tracking mechanism aiming at locally estimating the gradient of the smooth part of the sum-utility. To the best of our knowledge, this is the first provably convergent decentralized algorithm for Dictionary Learning and, more generally, bi-convex problems over (time-varying) (di)graphs.
Recommendations
- Distributed nonconvex constrained optimization over time-varying digraphs
- On the convergence of decentralized gradient descent
- Online learning over a decentralized network through ADMM
- DSA: decentralized double stochastic averaging gradient algorithm
- Decentralized learning for wireless communications and networking
Cites work
- rm K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
- Biclustering via sparse singular value decomposition
- Constrained Consensus and Optimization in Multi-Agent Networks
- Convergence of a Multi-Agent Projected Stochastic Gradient Algorithm for Non-Convex Optimization
- Decentralized Frank–Wolfe Algorithm for Convex and Nonconvex Problems
- Decomposition by Partial Linearization: Parallel Optimization of Multi-Agent Systems
- Decomposition into low-rank plus additive matrices for background/foreground separation: a review for a comparative evaluation with a large-scale dataset
- Dictionary Learning for Stereo Image Representation
- Dictionary Learning Over Distributed Models
- Distributed average consensus with least-mean-square deviation
- Distributed Dictionary Learning for Sparse Representation in Sensor Networks
- Distributed nonconvex constrained optimization over time-varying digraphs
- Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization
- scientific article; zbMATH DE number 6508162 (Why is no real title available?)
- scientific article; zbMATH DE number 6438182 (Why is no real title available?)
- Hybrid Random/Deterministic Parallel Algorithms for Convex and Nonconvex Big Data Optimization
- Learning the parts of objects by non-negative matrix factorization
- Learning Theory
- Non-Convex Distributed Optimization
- Non-negative matrix factorization with sparseness constraints
- Online learning for matrix factorization and sparse coding
- Parallel and distributed successive convex approximation methods for big-data optimization
- Parallel Selective Algorithms for Nonconvex Big Data Optimization
- Robust principal component analysis?
- Sparse principal component analysis via regularized low rank matrix approximation
- Variational Analysis
Cited in
(5)- Distributed algorithms for convex problems with linear coupling constraints
- Distributed cooperative learning over time-varying random networks using a gossip-based communication protocol
- Online matrix factorization for Markovian data and applications to network dictionary learning
- Decentralized learning for wireless communications and networking
- Decentralized Byzantine-resilient dictionary learning
This page was built for publication: Decentralized dictionary learning over time-varying digraphs
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5214231)
