Decentralized personalized federated learning: lower bounds and optimal algorithm for all personalization modes

DOI10.1016/J.EJCO.2022.100041MaRDI QIDQ6170035zbMATH OpenOpenAlexFDO

Authors Abdurakhmon Sadiev, Ekaterina Borodich, Aleksandr Beznosikov, Darina Dvinskikh, Savelii Chezhegov, Rachael Tappenden, M. Takáč, A. V. Gasnikov

Publication date 12 July 2023

Published in EURO Journal on Computational Optimization (Search for Journal in Brave)

Copyright license Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International

Full work available at URL https://arxiv.org/abs/2107.07190

zbMATH Keywords

accelerated algorithmsdecentralized optimizationdistributed optimizationfederated learninglower and upper bounds

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Convex programming (90C25) Nonlinear programming (90C30)

Abstract: This paper considers the problem of decentralized, personalized federated learning. For centralized personalized federated learning, a penalty that measures the deviation from the local model and its average, is often added to the objective function. However, in a decentralized setting this penalty is expensive in terms of communication costs, so here, a different penalty - one that is built to respect the structure of the underlying computational network - is used instead. We present lower bounds on the communication and local computation costs for this problem formulation and we also present provably optimal methods for decentralized personalized federated learning. Numerical experiments are presented to demonstrate the practical performance of our methods.

Recommendations

• Personalized federated learning: a clustered distributed co-meta-learning approach
• An Optimal Algorithm for Decentralized Finite-Sum Optimization
• Efficient and reliable overlay networks for decentralized federated learning
• On decentralized nonsmooth optimization
• DESTRESS: Computation-Optimal and Communication-Efficient Decentralized Nonconvex Finite-Sum Optimization

Cites work

• Advances and Open Problems in Federated Learning
• An Optimal Algorithm for Decentralized Finite-Sum Optimization
• Decentralized Accelerated Gradient Methods With Increasing Penalty Parameters
• Distributed Subgradient Methods for Multi-Agent Optimization
• Eigenvalues of the Laplacian of a graph^∗
• scientific article; zbMATH DE number 5454133 (Why is no real title available?)
• Introductory lectures on convex optimization. A basic course.
• Optimal convergence rates for convex distributed optimization in networks
• Understanding machine learning. From theory to algorithms

Cited in

(3)

• Decentralized multi-authority attribute-based inner-product FE: large universe and unbounded
• Personalized federated learning: a clustered distributed co-meta-learning approach
• One-point feedback for composite optimization with applications to distributed and federated learning

This page was built for publication: Decentralized personalized federated learning: lower bounds and optimal algorithm for all personalization modes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6170035)