On the Relation Between Recurrence and Ergodicity Properties in Denumerable Markov Decision Chains

DOI10.1287/moor.19.3.539zbMath0843.90127OpenAlexW2035161584MaRDI QIDQ4316544

Arie Hordijk, Rommert Dekker, Flora M. Spieksma

Publication date: 11 December 1994

Published in: Mathematics of Operations Research (Search for Journal in Brave)

Full work available at URL: http://repub.eur.nl/pub/2216

zbMATH Keywords

stabilityMarkov decision chain\(\mu\)-uniform geometric ergodicity\(\mu\)-uniform geometric recurrence

Mathematics Subject Classification ID

Markov and semi-Markov decision processes (90C40)

Related Items (14)

Risk-Sensitive Average Optimality for Discrete-Time Markov Decision Processes ⋮ Two-day appointment scheduling with patient preferences and geometric arrivals ⋮ The existence of sensitive optimal policies in two multi-dimensional queueing models ⋮ Kolmogorov forward equation and explosiveness in countable state Markov processes ⋮ Uniform ergodicity of continuous-time controlled Markov chains: a survey and new results ⋮ Zero-sum Markov games and worst-case optimal control of queueing systems ⋮ Perturbation analysis for denumerable Markov chains with application to queueing models ⋮ Blackwell optimality in the class of Markov policies for continuous-time controlled Markov chains ⋮ Nonzero-Sum Risk-Sensitive Stochastic Games on a Countable State Space ⋮ Taylor series expansions for stationary Markov chains ⋮ Characterization and sufficient conditions for normed ergodicity of Markov chains ⋮ Optimality of Mixed Policies for Average Continuous-Time Markov Decision Processes with Constraints ⋮ A novel use of value iteration for deriving bounds for threshold and switching curve optimal policies ⋮ On the reduction of total‐cost and average‐cost MDPs to discounted MDPs

This page was built for publication: On the Relation Between Recurrence and Ergodicity Properties in Denumerable Markov Decision Chains