Indices for families of competing Markov decision processes with influence (Q1313068)

In 1980 Whittle (a) explained Gittins indices for multiarmed bandit processes (MBP) in terms of equivalent retirement rewards; and (b) gave conditions under which there is an optimal policy of index type for MBP, where each arm has its own decision apparatus. In the same year Nash proved indexation for a further class of MBP in which the current states of all of the arms can influence the reward earned from the active one. In the present paper the author develops analogues of (a) and (b) for Nash's model. He then illustrates the theory by examining stoppable MBP, where each arm has two actions: continue and stop.