Distributed networked learning with correlated data

DOI10.1016/J.AUTOMATICA.2021.110134MaRDI QIDQ2071980zbMATH OpenOpenAlexFDO

Authors Lingzhou Hong, Ceyhun Eksin, Alfredo García

Publication date 31 January 2022

Published in Automatica (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1910.12783

zbMATH Keywords

large scale optimization problems and methodslearning theorymulti-agent systemsnetwork-based computing systemsparameter and state estimationstatistical analysis

Mathematics Subject Classification ID

Large-scale systems (93A15) Multi-agent systems (93A16) Networked control (93B70) Control/observation systems involving computers (process control, etc.) (93C83)

Abstract: We consider a distributed estimation method in a setting with heterogeneous streams of correlated data distributed across nodes in a network. In the considered approach, linear models are estimated locally (i.e., with only local data) subject to a network regularization term that penalizes a local model that differs from neighboring models. We analyze computation dynamics (associated with stochastic gradient updates) and information exchange (associated with exchanging current models with neighboring nodes). We provide a finite-time characterization of convergence of the weighted ensemble average estimate and compare this result to federated learning, an alternative approach to estimation wherein a single model is updated by locally generated gradient updates. This comparison highlights the trade-off between speed vs precision: while model updates take place at a faster rate in federated learning, the proposed networked approach to estimation enables the identification of models with higher precision. We illustrate the method's general applicability in two examples: estimating a Markov random field using wireless sensor networks and modeling prey escape behavior of flocking birds based on a publicly available dataset.

Recommendations

• Adaptation, learning, and optimization over networks
• Distributed estimation for parameter in heterogeneous linear time-varying models with observations at network sensors
• Distributed cooperative learning over time-varying random networks using a gossip-based communication protocol
• Distributed linear regression by averaging
• Online supervised learning with distributed features over multiagent system

Cites work

• A Collaborative Training Algorithm for Distributed Learning
• A Distributed Algorithm for Least Squares Solutions
• A Distributed Algorithm for Solving a Linear Algebraic Equation
• Asynchronous Distributed Algorithms for Solving Linear Algebraic Equations
• Communication-efficient algorithms for statistical optimization
• Distributed Network Optimization With Heuristic Rational Agents
• Distributed Subgradient Methods for Multi-Agent Optimization
• Distributed weighted least-squares estimation with fast convergence for large-scale systems
• Divide and conquer kernel ridge regression: a distributed algorithm with minimax optimal rates
• Ensemble approaches for regression: a survey
• EXTRA: an exact first-order algorithm for decentralized consensus optimization
• Generalized Least Squares Model Averaging
• scientific article; zbMATH DE number 1600999 (Why is no real title available?)
• scientific article; zbMATH DE number 5060482 (Why is no real title available?)
• scientific article; zbMATH DE number 3065414 (Why is no real title available?)
• Least Squares Model Averaging
• Multitask Diffusion Adaptation Over Networks
• Multitask Diffusion Adaptation Over<?Pub _newline ?>Asynchronous Networks
• The Fokker-Planck equation. Methods of solutions and applications.

Cited in

(4)

• Distributed Censored Regression Over Networks
• Learning from networked examples
• Distributed learning for random vector functional-link networks
• Non-convex composite federated learning with heterogeneous data

This page was built for publication: Distributed networked learning with correlated data

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2071980)