A policy iteration algorithm for nonzero-sum stochastic impulse games

DOI10.1051/PROC/201965027MaRDI QIDQ4967861zbMATH OpenOpenAlexWikidataFDO

Authors René Aïd, Francisco Bernal, Diego Zabaljauregui, J. P. Zubelli, Mohamed Mnif

Publication date 11 July 2019

Published in ESAIM: Proceedings and Surveys (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://arxiv.org/abs/1803.05830

zbMATH Keywords

Howard's algorithmNash equilibriumnonzero-sum gamepolicy iterationquasi-variational inequalitystochastic impulse game

Mathematics Subject Classification ID

Variational inequalities (49J40) Impulsive optimal control problems (49N25) Differential games and control (49N70)

Abstract: This work presents a novel policy iteration algorithm to tackle nonzero-sum stochastic impulse games arising naturally in many applications. Despite the obvious impact of solving such problems, there are no suitable numerical methods available, to the best of our knowledge. Our method relies on the recently introduced characterization of the value functions and Nash equilibrium via a system of quasi-variational inequalities. While our algorithm is heuristic and we do not provide a convergence analysis, numerical tests show that it performs convincingly in a wide range of situations, including the only analytically solvable example available in the literature at the time of writing.

Recommendations

• A fixed-point policy-iteration-type algorithm for symmetric nonzero-sum stochastic impulse control games
• A policy iteration algorithm for zero-sum stochastic games with mean payoff
• A policy iteration algorithm for nonzero-sum games with unknown models
• Policy iteration algorithms for zero-sum stochastic differential games with long-run average payoff criteria
• A policy iteration method for mean field games
• scientific article; zbMATH DE number 1944075
• Nonzero-sum stochastic games and mean-field games with impulse controls
• A policy-improvement type algorithm for solving zero-sum two-person stochastic games of perfect information
• scientific article; zbMATH DE number 3854935

Cites work

• A policy iteration algorithm for fixed point problems with nonexpansive operators
• Functional equation in the theory of dynamic programming. XVI: An equation involving iterates
• scientific article; zbMATH DE number 3148886 (Why is no real title available?)
• scientific article; zbMATH DE number 5730451 (Why is no real title available?)
• scientific article; zbMATH DE number 1897413 (Why is no real title available?)
• Nonzero-sum stochastic differential games with impulse controls: a verification theorem with applications
• Some Convergence Results for Howard's Algorithm
• Stochastic differential games involving impulse controls and double-obstacle quasi-variational inequalities

Cited in

(5)

• Off-policy based adaptive dynamic programming method for nonzero-sum games on discrete-time system
• A fixed-point policy-iteration-type algorithm for symmetric nonzero-sum stochastic impulse control games
• A policy iteration method for mean field games
• Online solution of nonlinear two‐player zero‐sum games using synchronous policy iteration
• Nonzero-Sum Stochastic Impulse Games with an Application in Competitive Retail Energy Markets

This page was built for publication: A policy iteration algorithm for nonzero-sum stochastic impulse games

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4967861)