The set-indexed bandit problem.
This paper proves the existence of an optimal solution to the bandit problem formulated in the framework of set-indexed processes, i.e., the existence of a set-valued increasing path \(\xi \) maximizing the functional \(E(\int _{[0,1]}X_\xi \,\text dV_t)\) with appropriate processes \(X\) and \(V\). The basic idea of the proof is to fuzzify set-valued increasing paths, and then to prove for a fuzzy extension of the above functional that an optimal solution exists and is assumed for fuzzy increasing paths that correspond to the usual (crisp) increasing paths. The concluding part of the proof has been inspired by the proof of an analogical result for continuous-time processes, given by \textit{R. C. Dalang} [Ann. Probab. 18, 218--225 (1990; Zbl 0713.60054)], but the extension to set-indexed processes requires three preparatory steps. First, fuzzy stopping sets and fuzzy increasing paths are defined, and their basic properties are established. Those concepts rely on the usual relationship between random and fuzzy sets, although the formal definition of fuzzy sets in the paper differs from what is common in the fuzzy set theory. Second, the integration on collections of compact connected subsets of compact complete metric spaces is defined and used to extend the considered functional to fuzzy increasing paths. Finally, an extension of the Baxter-Chacon topology is introduced [\textit{J.\ R.\ Baxter} and \textit{R.\ V.\ Chacon}, Z.\ Wahrscheinlichkeitstheorie Verw.\ Geb.\ 40, 169--181 (1977; Zbl 0349.60048)], and the compactness of fuzzy stopping sets and fuzzy increasing paths in that topology is shown. The result presented in the paper is in a sense a continuation of deep results concerning set-indexed processed, many of which have been collected in a recent monograph by \textit{G.\ Ivanoff} and \textit{E.\ Merzbach} [``Set-indexed martingales (2000; Zbl 0948.60039)]. Differently to that monograph, the paper is very difficult to read in detail because it contains numerous sources of unintentional confusion (concepts used in a more general setting than in which they were defined, a statement proved in a lemma but forgotten in its formulation, various misprints in notation, and an undiscoverable misprint in a key proposition concerning integration on set collections). Nevertheless, it is well structured and all important steps of the proof are clearly conveyed.
- Compactness of stopping times
- Continuous multi-armed bandits and multiparameter processes
- scientific article; zbMATH DE number 4064879 (Why is no real title available?)
- scientific article; zbMATH DE number 3757085 (Why is no real title available?)
- scientific article; zbMATH DE number 1348602 (Why is no real title available?)
- scientific article; zbMATH DE number 194374 (Why is no real title available?)
- scientific article; zbMATH DE number 3234211 (Why is no real title available?)
- Multi-armed bandits in discrete and continuous time
- On different topologies for set-indexing collections
- Optimal Stopping of Two-Parameter Processes on Nonstandard Probability Spaces
- Random censoring in set-indexed survival analysis
- Randomization in the two-armed bandit problem
- Stochastic control of two-parameter processes application:the two-armed bandit problem
- The optional sampling theorem for martingales indexed by directed sets
- The optional sampling theorem for processes indexed by a partially ordered set
This page was built for publication: The set-indexed bandit problem.
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2574504)
