On monotone optimal decision rules and the stay-on-a-winner rule for the two-armed bandit (Q1821704)

Consider the following optimization problem: Find a decision rule \(\delta\) such that \(w(x,\delta (x))=\max_{a}w(x,a)\) for all x under the constraint \(\delta\) (x)\(\in D(x)\). We give conditions for the existence of monotone optimal decision rules \(\delta\). The term 'monotone' is used in a general sense. The well-known stay-on-a-winner rules for the two- armed bandit can be characterized as monotone decision rules by including the stage number into x and using a special ordering on x. This enables us to give simple conditions for the existence of optimal rules that are stay-on-a-winner rules. We extend results of \textit{D. A. Berry} [Ann. Math. Stat. 43, 871-897 (1972; Zbl 0258.62013)] and \textit{D. Kalin} and \textit{R. Theodorescu} [Math. Operations-Forsch. Stat., Ser. Optimization 13, 469-472 (1982; Zbl 0505.90080)] to the case of dependent arms.