Multi-armed bandit processes with optimal selection of the operating times

DOI10.1007/BF02595405MaRDI QIDQ2387146zbMATH OpenOpenAlexFDO

Authors Pilar Ibarrola Muñoz, Ricardo Vélez Ibarrola

Publication date 1 September 2005

Published in Test (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1007/bf02595405

zbMATH Keywords

Gittins indexmulti-armed bandit processes

Mathematics Subject Classification ID

Stopping times; optimal stopping problems; gambling theory (60G40) Sequential statistical analysis (62L10) Stochastic scheduling theory in operations research (90B36) Markov and semi-Markov decision processes (90C40)

Review text

A multi-armed Bandit Problem is considered such that at each decision epoch it is to be decided the next project to be undertaken and the span of time to be spent in this project, instead of reconsidering the new project at each stage. This extended model, inspired in sequentially planned decision procedures [\textit{W. Schmitz} ``Optimal sequentially planned decision procedures. Lect. Notes Stat. 79. New York: Springer-Verlag (1993; Zbl 0771.62057)], is formulated in Section 1 and tries to exploit the reduction of costs produced by longer periods dedicated to the same activity. Following the method by \textit{P. Whittle} [J. R. Stat. Soc., Ser. B. 42, 143--149 (1980; Zbl 0439.90096), Section 2 introduces a retirement option with a variable reward \(M\), and Section 3 extends Gittins indexes to this case. Another relevant conclusion is that the optimal period of activity for each project does not depend on the retirement reward \(M\). Finally, we show that the optimal strategy is to choose the project with the highest Gittins index.

Recommendations

• Multi-armed bandits in discrete and continuous time
• Continuous multi-armed bandits and multiparameter processes
• Finite-time analysis of the multiarmed bandit problem
• Discrete multiarmed bandits and multiparameter processes
• scientific article; zbMATH DE number 2156732
• On the Optimal Reward Function of the Continuous Time Multiarmed Bandit Problem
• Optimal stopping problems for multiarmed bandit processes with arms' independence
• Optimal policies for observing time series and related restless bandit problems
• Optimal strategies for families of alternative bandit processes

Cites work

• A lemma on the multiarmed bandit problem
• scientific article; zbMATH DE number 3889341 (Why is no real title available?)
• scientific article; zbMATH DE number 3687126 (Why is no real title available?)
• scientific article; zbMATH DE number 3474804 (Why is no real title available?)
• scientific article; zbMATH DE number 3638998 (Why is no real title available?)
• scientific article; zbMATH DE number 194374 (Why is no real title available?)
• Multi-armed bandits with switching penalties
• On a sufficient condition for superprocesses due to whittle
• On the Gittins index for multiarmed bandits
• Optimal sequentially planned decision procedures. With the assistance of Günter Duscha, Josef Lübbert and Thomas Meyerthole
• Switching Costs and the Gittins Index

Cited in

(6)

• Optimal stopping problems for multiarmed bandit processes with arms' independence
• Robust control of the multi-armed bandit problem
• scientific article; zbMATH DE number 4087408 (Why is no real title available?)
• Multi-armed bandits under general depreciation and commitment
• Analyzing bandit-based adaptive operator selection mechanisms
• A perpetual search for talents across overlapping generations: a learning process

This page was built for publication: Multi-armed bandit processes with optimal selection of the operating times

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2387146)