Optimal stopping of Markov chains and three abstract optimization problems

DOI10.1080/17442508.2010.514051MaRDI QIDQ3108369zbMATH OpenOpenAlexFDO

Authors Isaac Sonin

Publication date 3 January 2012

Published in Stochastics (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1080/17442508.2010.514051

zbMATH Keywords

Markov chainGittins indexoptimal stoppingthe elimination algorithm

Mathematics Subject Classification ID

Computational methods in Markov chains (60J22) Numerical analysis or methods applied to Markov chains (65C40) Stopping times; optimal stopping problems; gambling theory (60G40) Markov and semi-Markov decision processes (90C40)

Recommendations

• A generalized Gittins index for a Markov chain and its recursive calculation
• A \((2/3)n^{3}\) fast-pivoting algorithm for the Gittins index and optimal stopping of a Markov chain
• The elimination algorithm for the problem of optimal stopping
• General Gittins index processes in discrete time.
• On optimal stopping of random sequences modulated by Markov chain

Cites work

• A generalized Gittins index for a Markov chain and its recursive calculation
• A stochastic representation theorem with applications to optimization and obstacle problems.
• Continue, quit, restart probability model
• Index Policies for Stochastic Search in a Forest with an Application to R&D Project Management
• Irreversible decisions under uncertainty. Optimal stopping made easy
• Optimal Sequencing and Resource Allocation in Research and Development Projects
• The Multi-Armed Bandit Problem: Decomposition and Computation
• The state reduction and related algorithms and their applications to the study of Markov chains, graph theory, and the optimal stopping problem

Cited in

(9)

• On Optimal Multiple Stopping of Markov Sequences
• DES and RES processes and their explicit solutions
• Optimal activation of halting multi‐armed bandit models
• Continue, quit, restart probability model
• scientific article; zbMATH DE number 5066282 (Why is no real title available?)
• On Playing Golf with Two Balls
• Optimal starting—stopping problems for markov–feller processes
• A generalized Gittins index for a Markov chain and its recursive calculation
• Multi-armed bandits under general depreciation and commitment

This page was built for publication: Optimal stopping of Markov chains and three abstract optimization problems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3108369)