Discrete multiarmed bandits and multiparameter processes (Q1317211)

The author reformulates the multiarmed bandit problem in discrete time as an optimal stochastic control problem for a multiparameter process. Within this framework, the dynamic allocation index, the so-called Gittins index, becomes a multiparameter process, and it is shown how it leads to optimal solutions. The main advantage of such an approach is that it provides a convenient and elegant representation of switching strategies by using the notion of optimal increasing paths or strategies over a partially ordered set.