Scalable Reinforcement Learning for Multi-Agent Networked Systems

DOI10.1287/OPRE.2021.2226MaRDI QIDQ6330559arXivFDO

Authors Guannan Qu, Adam Wierman, Na Li

Publication date 5 December 2019

Published in arXiv

Full work available at URL https://doi.org/10.1287/OPRE.2021.2226

Mathematics Subject Classification ID

Deterministic network models in operations research (90B10)

arXiv classification

cs.AIcs.LGmath.OC

Abstract: We study reinforcement learning (RL) in a setting with a network of agents whose states and actions interact in a local manner where the objective is to find localized policies such that the (discounted) global reward is maximized. A fundamental challenge in this setting is that the state-action space size scales exponentially in the number of agents, rendering the problem intractable for large networks. In this paper, we propose a Scalable Actor Critic (SAC) framework that exploits the network structure and finds a localized policy that is an $O(h{o}^{kappa})$-approximation of a stationary point of the objective for some $hoin(0,1)$, with complexity that scales with the local state-action space size of the largest $kappa$-hop neighborhood of the network. We illustrate our model and approach using examples from wireless communication, epidemics and traffic.

This page was built for publication: Scalable Reinforcement Learning for Multi-Agent Networked Systems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6330559)