Learning Optimal Policies in Potential Mean Field Games: Smoothed Policy Iteration Algorithms

DOI10.1137/22M1539861MaRDI QIDQ6148451zbMATH OpenOpenAlexWikidataFDO

Authors Qing Tang, Jiahao Song

Publication date 7 February 2024

Published in SIAM Journal on Control and Optimization (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2212.04791

zbMATH Keywords

mean field gamesnumerical methodspolicy iterationlearning in games

Mathematics Subject Classification ID

Finite difference methods for initial value and initial-boundary value problems involving PDEs (65M06) Rationality and learning in game theory (91A26) Mean field games (aspects of game theory) (91A16)

Abstract: We introduce two Smoothed Policy Iteration algorithms ( extbf{SPI}s) as rules for learning policies and methods for computing Nash equilibria in second order potential Mean Field Games (MFGs). Global convergence is proved if the coupling term in the MFG system satisfy the Lasry Lions monotonicity condition. Local convergence to a stable solution is proved for system which may have multiple solutions. The convergence analysis shows close connections between extbf{SPI}s and the Fictitious Play algorithm, which has been widely studied in the MFG literature. Numerical simulation results based on finite difference schemes are presented to supplement the theoretical analysis.

Recommendations

• A policy iteration method for mean field games
• Learning in mean field games: The fictitious play
• Generalized conditional gradient and learning in potential mean field games
• A General Framework for Learning Mean-Field Games
• Rates of convergence for the policy iteration method for mean field games systems

Cites work

• scientific article; zbMATH DE number 3148886 (Why is no real title available?)
• scientific article; zbMATH DE number 1233801 (Why is no real title available?)
• scientific article; zbMATH DE number 3277871 (Why is no real title available?)
• A policy iteration method for mean field games
• Fictitious play property for games with identical interests
• Finite mean field games: fictitious play and convergence to a first order continuous mean field game
• Global properties of transition probabilities of singular diffusions
• Large-Population Cost-Coupled LQG Problems With Nonuniform Agents: Individual-Mass Behavior and Decentralized $\varepsilon$-Nash Equilibria
• Learning in mean field games: The fictitious play
• Lipschitz regularity for viscous Hamilton-Jacobi equations with \(L^p\) terms
• Long time behavior and turnpike solutions in mildly non-monotone mean field games
• Maximal \(L^q\)-regularity for parabolic Hamilton-Jacobi equations and applications to mean field games
• Mean field games
• Mean field games and mean field type control theory
• Mean field games of controls: finite difference approximations
• Mean field games: convergence of a finite difference method
• Mean field games: numerical methods
• On monotonicity conditions for mean field games
• On non-uniqueness and uniqueness of solutions in finite-horizon Mean Field Games
• Policy iteration method for time-dependent mean field games systems with non-separable Hamiltonians
• Probabilistic theory of mean field games with applications I. Mean field FBSDEs, control, and games
• Rates of convergence for the policy iteration method for mean field games systems
• Semiconcave functions, Hamilton-Jacobi equations, and optimal control
• Short-time existence for a general backward-forward parabolic system arising from mean-field games
• Stable solutions in potential mean field game systems
• Time-dependent focusing mean-field games: the sub-critical case

Cited in

(1)

• Generalized conditional gradient and learning in potential mean field games

This page was built for publication: Learning Optimal Policies in Potential Mean Field Games: Smoothed Policy Iteration Algorithms

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6148451)