Fictitious play in stochastic games (Q2472192)

An explicit example of a stochastic game is considered, with two players, two states and two actions, and with special structure such as single controller condition, irreducibility, and state-independent transitions. This has a unique equilibrium in stationary strategies. It is shown that even in this apparently simple situation, fictitious play, wherein a player always plays her best response to the opponent's empirical policy, does not converge.