On Gittins' index theorem in continuous time (Q2642040)

The Gittins' index theorem is proved for dynamic allocation problems of the multi-armed bandit type. The authors formalize their allocation problem as a multiparameter control problem. They construct the new approach to Gittins' index theorem that is based on a characteristic representation property which relates the Gittins index to the accumulated future expected rewards from a given project. This property, in conjunction with a novel partial-integration argument, not only allows for a comparably short proof of Gittins' index theorem under minimal asumptions, but it actually allows to give necessary and sufficient conditions for optimality. Another important point is the lower semi-continuity of Gittins index processes. This new regularity result is crucial not only for the continuous-time construction of Gittins indices as proper optional processes but it turns out to be indispensable even for the existence of optimal allocation strategies.