Independent learning in stochastic games

DOI10.4171/ICM2022/152OpenAlexW4389775604MaRDI QIDQ6200215FDOQ6200215

Authors: Asuman Ozdaglar, Muhammed O. Sayin, Kaiqing Zhang 

Publication date: 22 March 2024

Published in: International Congress of Mathematicians (Search for Journal in Brave)

Abstract: Reinforcement learning (RL) has recently achieved tremendous successes in many artificial intelligence applications. Many of the forefront applications of RL involve multiple agents, e.g., playing chess and Go games, autonomous driving, and robotics. Unfortunately, the framework upon which classical RL builds is inappropriate for multi-agent learning, as it assumes an agent's environment is stationary and does not take into account the adaptivity of other agents. In this review paper, we present the model of stochastic games for multi-agent learning in dynamic environments. We focus on the development of simple and independent learning dynamics for stochastic games: each agent is myopic and chooses best-response type actions to other agents' strategy without any coordination with her opponent. There has been limited progress on developing convergent best-response type independent learning dynamics for stochastic games. We present our recently proposed simple and independent learning dynamics that guarantee convergence in zero-sum stochastic games, together with a review of other contemporaneous algorithms for dynamic multi-agent learning in this setting. Along the way, we also reexamine some classical results from both the game theory and RL literature, to situate both the conceptual contributions of our independent learning dynamics, and the mathematical novelties of our analysis. We hope this review paper serves as an impetus for the resurgence of studying independent and natural learning dynamics in game theory, for the more challenging settings with a dynamic environment.

Full work available at URL: https://arxiv.org/abs/2111.11743

Recommendations

• Best-response dynamics in zero-sum stochastic games
• 10.1162/1532443041827880
• Fictitious play in zero-sum stochastic games
• Multiagent learning using a variable learning rate

zbMATH Keywords

stochastic gamesreinforcement learninglearning in games

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Stochastic games, stochastic differential games (91A15) Rationality and learning in game theory (91A26)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• 10.1162/153244303765208377
• A Theory of Dynamic Oligopoly, I: Overview and Quantity Competition with Large Fixed Costs
• A Theory of Dynamic Oligopoly, II: Price Competition, Kinked Demand Curves, and Edgeworth Cycles
• A \(2 \times 2\) game without the fictitious play property
• A weakened form of fictitious play in two-person zero-sum games
• Adaptive and sophisticated learning in normal form games
• Adaptive game playing using multiplicative weights
• An iterative method of solving a game
• Asynchronous stochastic approximation and Q-learning
• Asynchronous stochastic approximation with differential inclusions
• Best-response dynamics in zero-sum stochastic games
• Composable and efficient mechanisms
• Consistency and cautious fictitious play
• Decentralized Q-Learning for Stochastic Teams and Games
• Distributed dynamic programming
• Fictitious play in one-against-all multi-player games
• Fictitious play in \(2\times n\) games
• Fictitious play property for games with identical interests
• Individual Q-Learning in Normal Form Games
• Intrinsic robustness of the price of anarchy
• Learning in games with strategic complementarities revisited
• Learning mixed equilibria
• Multi-agent reinforcement learning: a selective overview of theories and algorithms
• Nash equilibrium and the evolution of preferences
• On the Global Convergence of Stochastic Fictitious Play
• On the rate of convergence of continuous-time fictitious play
• Online learning and online convex optimization
• Potential games
• Prediction, Learning, and Games
• Quantal response equilibria for normal form games
• REINFORCEMENT LEARNING IN MARKOVIAN EVOLUTIONARY GAMES
• Stochastic Approximations and Differential Inclusions
• Stochastic Games
• Stochastic approximation methods for constrained and unconstrained systems
• Stochastic approximation. A dynamical systems viewpoint.
• The weighted majority algorithm
• \({\mathcal Q}\)-learning

This page was built for publication: Independent learning in stochastic games

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6200215)