Sequential decomposition and policy iteration schemes for \(M\)-player games with partial weak coupling (Q1190505)

The paper formulates two general classes of \(M\)-player deterministic and stochastic nonzero-sum games where the players can be placed into two groups such that there are strong interactions within each group and a weak interaction between the two groups. This weak interaction is characterized in terms of a small parameter \(\varepsilon\) which, when set equal to zero, leads to two independent nonzero-sum games. Under the Nash equilibrium solution concept both within and in between the groups, the authors study the merits of an iterative method for the construction of the equilibrium by solving simpler problems at each stage of the iteration. In this iterative scheme, the zeroth order solution is the Nash equilibrium of the two independent games obtained by setting \(\varepsilon=0\), whereas the higher-order solutions are Nash equilibria of quadratic games, even though the original problem may have non- quadratic cost functions. It is shown that the magnitude of \(\varepsilon\) determines the convergence, or otherwise, of policy iteration schemes applied to such games. Also studied is the effect of relaxation techniques on the convergence of policy iteration. The analysis is carried out in both finite- and infinite-dimensional decision spaces, with the latter allowing the consideration of (nonlinear) differential games where the players have access to open-loop information. Another extension considered in the paper are static stochastic games, where the players are endowed with correlated (stochastic) measurements on a given state of nature which affects the cost functions of all players. Here again the decomposition of the original weakly coupled game into smaller size (stochastic) games is studied.