Average optimality for continuous-time Markov decision processes in Polish spaces

DOI10.1214/105051606000000105zbMath1160.90010arXivmath/0607098OpenAlexW2026850974MaRDI QIDQ997948

Xianping Guo, Ulrich Rieder

Publication date: 8 August 2007

Published in: The Annals of Applied Probability (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/math/0607098

zbMATH Keywords

optimal stationary policyaverage rewardoptimality inequalitygeneral state spacesemimartingale characterization

Mathematics Subject Classification ID

Optimal stochastic control (93E20) Markov and semi-Markov decision processes (90C40)

Related Items

Kolmogorov's Equations for Jump Markov Processes and Their Applications to Control Problems ⋮ Estimate the exponential convergence rate of \(f\)-ergodicity via spectral gap ⋮ Uniform ergodicity of continuous-time controlled Markov chains: a survey and new results ⋮ Average stochastic games for continuous-time jump processes ⋮ Risk sensitive control of pure jump processes on a general state space ⋮ The policy iteration algorithm for average continuous control of piecewise deterministic Markov processes ⋮ Strong \(n\)-discount and finite-horizon optimality for continuous-time Markov decision processes ⋮ Average sample-path optimality for continuous-time Markov decision processes in Polish spaces ⋮ A note on the existence of optimal stationary policies for average Markov decision processes with countable states ⋮ Construction and regularity of transition functions on Polish spaces under measurability conditions ⋮ Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces ⋮ New average optimality conditions for semi-Markov decision processes in Borel spaces ⋮ Verifiable conditions for average optimality of continuous-time Markov decision processes ⋮ Discounted continuous-time constrained Markov decision processes in Polish spaces ⋮ Average optimality for continuous-time Markov decision processes with a policy iteration approach ⋮ Average optimality inequality for continuous-time Markov decision processes in Polish spaces ⋮ Policy iteration for continuous-time average reward Markov decision processes in Polish spaces ⋮ Continuous-time Markov decision processes with \(n\)th-bias optimality criteria ⋮ On solutions of Kolmogorov's equations for nonhomogeneous jump Markov processes ⋮ Nonzero-sum games for continuous-time jump processes under the expected average payoff criterion ⋮ Variance minimization for continuous-time Markov decision processes: two approaches ⋮ Computable approximations for average Markov decision processes in continuous time ⋮ A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder) ⋮ New sufficient conditions for average optimality in continuous-time Markov decision processes ⋮ New discount and average optimality conditions for continuous-time Markov decision processes ⋮ The Vanishing Discount Approach for the Average Continuous Control of Piecewise Deterministic Markov Processes ⋮ Optimality of Mixed Policies for Average Continuous-Time Markov Decision Processes with Constraints ⋮ Absorbing Continuous-Time Markov Decision Processes with Total Cost Criteria ⋮ Average optimality for continuous-time Markov decision processes under weak continuity conditions ⋮ Unnamed Item

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Continuous time Markov decision programming with average reward criterion and unbounded reward rate
• Continuous-time Markov chains. An applications-oriented approach
• Continuous time control of Markov processes on an arbitrary state space: Discounted rewards
• Measurable selection theorems for optimization problems
• Continuous-time controlled Markov chains.
• Equivalence of exponential ergodicity and \(L^ 2\)-exponential convergence for Markov chains.
• Exponential and uniform ergodicity of Markov processes
• Computable exponential convergence rates for stochastically ordered Markov processes
• On maximal rewards and \(\varepsilon\)-optimal policies in continuous time Markov decision chains
• Finite state continuous time Markov decision processes with an infinite planning horizon
• Stability of Markovian processes III: Foster–Lyapunov criteria for continuous-time processes
• On Homogeneous Markov Models with Continuous Time and Finite or Countable State Space
• Generalized potlatch and smoothing processes
• Criteria for ergodicity, exponential ergodicity and strong ergodicity of Markov processes
• Some estimates of the rate of convergence for birth and death processes
• Bias Optimality in Controlled Queueing Systems
• Zero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates
• A note on bias optimality in controlled queueing systems
• A note on optimality conditions for continuous-time Markov decision processes with average cost criterion
• Average Optimality in Markov Control Processes via Discounted-Cost Problems and Linear Programming
• Nondiscounted Continuous Time Markovian Decision Process with Countable State Space
• Drift and monotonicity conditions for continuous-time controlled markov chains with an average criterion
• Optimal Control of Ergodic Continuous-Time Markov Chains with Average Sample-Path Rewards
• On the Integro-Differential Equations of Purely Discontinuous Markoff Processes