Federated Optimization Under Intermittent Client Availability

DOI10.1287/IJOC.2022.0057arXiv2002.07399MaRDI QIDQ6202343FDOQ6202343

Authors: Chaoyue Niu, Zhenzhe Zheng, Shaojie Tang, Fan Wu, Yanghe Feng, Guihai Chen 

Publication date: 26 March 2024

Published in: INFORMS Journal on Computing (Search for Journal in Brave)

Abstract: Federated learning is a new distributed machine learning framework, where a bunch of heterogeneous clients collaboratively train a model without sharing training data. In this work, we consider a practical and ubiquitous issue when deploying federated learning in mobile environments: intermittent client availability, where the set of eligible clients may change during the training process. Such intermittent client availability would seriously deteriorate the performance of the classical Federated Averaging algorithm (FedAvg for short). Thus, we propose a simple distributed non-convex optimization algorithm, called Federated Latest Averaging (FedLaAvg for short), which leverages the latest gradients of all clients, even when the clients are not available, to jointly update the global model in each iteration. Our theoretical analysis shows that FedLaAvg attains the convergence rate of $O(E^{1/2}/(N^{1/4}{T}^{1/2}))$, achieving a sublinear speedup with respect to the total number of clients. We implement FedLaAvg along with several baselines and evaluate them over the benchmarking MNIST and Sentiment140 datasets. The evaluation results demonstrate that FedLaAvg achieves more stable training than FedAvg in both convex and non-convex settings and indeed reaches a sublinear speedup.

Full work available at URL: https://arxiv.org/abs/2002.07399

Recommendations

• Federated learning for minimizing nonsmooth convex loss functions
• Advances and Open Problems in Federated Learning
• Federated primal dual fixed point algorithm
• Efficient and reliable overlay networks for decentralized federated learning
• Gradient-free federated learning methods with \(l_1\) and \(l_2\)-randomization for non-smooth convex stochastic optimization problems

zbMATH Keywords

nonconvex optimizationfederated learningclient availability

Mathematics Subject Classification ID

Mathematical programming (90Cxx)

This page was built for publication: Federated Optimization Under Intermittent Client Availability

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6202343)