Learning Optimal Policies in Potential Mean Field Games: Smoothed Policy Iteration Algorithms

DOI10.1137/22M1539861arXiv2212.04791OpenAlexW4391170322WikidataQ129504108 ScholiaQ129504108MaRDI QIDQ6148451FDOQ6148451

Authors: Qing Tang, Jiahao Song 

Publication date: 7 February 2024

Published in: SIAM Journal on Control and Optimization (Search for Journal in Brave)

Abstract: We introduce two Smoothed Policy Iteration algorithms ( extbf{SPI}s) as rules for learning policies and methods for computing Nash equilibria in second order potential Mean Field Games (MFGs). Global convergence is proved if the coupling term in the MFG system satisfy the Lasry Lions monotonicity condition. Local convergence to a stable solution is proved for system which may have multiple solutions. The convergence analysis shows close connections between extbf{SPI}s and the Fictitious Play algorithm, which has been widely studied in the MFG literature. Numerical simulation results based on finite difference schemes are presented to supplement the theoretical analysis.

Full work available at URL: https://arxiv.org/abs/2212.04791

Recommendations

• A policy iteration method for mean field games
• Learning in mean field games: The fictitious play
• Generalized conditional gradient and learning in potential mean field games
• A General Framework for Learning Mean-Field Games
• Rates of convergence for the policy iteration method for mean field games systems

zbMATH Keywords

mean field gamesnumerical methodspolicy iterationlearning in games

Mathematics Subject Classification ID

Finite difference methods for initial value and initial-boundary value problems involving PDEs (65M06) Rationality and learning in game theory (91A26) Mean field games (aspects of game theory) (91A16)

Cites Work

• Title not available (Why is that?)
• Mean field games
• Mean field games: convergence of a finite difference method
• Large-Population Cost-Coupled LQG Problems With Nonuniform Agents: Individual-Mass Behavior and Decentralized $\varepsilon$-Nash Equilibria
• Mean field games and mean field type control theory
• Mean field games: numerical methods
• Title not available (Why is that?)
• Semiconcave functions, Hamilton-Jacobi equations, and optimal control
• Title not available (Why is that?)
• Fictitious play property for games with identical interests
• Probabilistic theory of mean field games with applications I. Mean field FBSDEs, control, and games
• Global properties of transition probabilities of singular diffusions
• Learning in mean field games: The fictitious play
• Stable solutions in potential mean field game systems
• Time-dependent focusing mean-field games: the sub-critical case
• Lipschitz regularity for viscous Hamilton-Jacobi equations with \(L^p\) terms
• A policy iteration method for mean field games
• Rates of convergence for the policy iteration method for mean field games systems
• On non-uniqueness and uniqueness of solutions in finite-horizon Mean Field Games
• Long time behavior and turnpike solutions in mildly non-monotone mean field games
• On monotonicity conditions for mean field games
• Mean field games of controls: finite difference approximations
• Short-time existence for a general backward-forward parabolic system arising from mean-field games
• Finite mean field games: fictitious play and convergence to a first order continuous mean field game
• Maximal \(L^q\)-regularity for parabolic Hamilton-Jacobi equations and applications to mean field games
• Policy iteration method for time-dependent mean field games systems with non-separable Hamiltonians

Cited In (1)

• Generalized conditional gradient and learning in potential mean field games

This page was built for publication: Learning Optimal Policies in Potential Mean Field Games: Smoothed Policy Iteration Algorithms

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6148451)