Average optimality for Markov decision processes in borel spaces: a new condition and approach

DOI10.1239/jap/1152413725zbMath1121.90122OpenAlexW1994038771MaRDI QIDQ3410916

Quanxin Zhu, Xianping Guo

Publication date: 16 November 2006

Published in: Journal of Applied Probability (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1239/jap/1152413725

zbMATH Keywords

optimal stationary policydiscrete-time Markov decision processaverage expected criterionaverage optimality inequality

Mathematics Subject Classification ID

Optimal stochastic control (93E20) Markov and semi-Markov decision processes (90C40)

Related Items (21)

Markov Decision Processes with Variance Minimization: A New Condition and Approach ⋮ Bias optimality and strong \(n\) \((n= -1,0)\) discount optimality for Markov decision processes ⋮ A semimartingale characterization of average optimal stationary policies for Markov decision processes ⋮ The policy iteration algorithm for average continuous control of piecewise deterministic Markov processes ⋮ Two-person zero-sum stochastic games with varying discount factors ⋮ Another set of verifiable conditions for average Markov decision processes with Borel spaces ⋮ New average optimality conditions for semi-Markov decision processes in Borel spaces ⋮ Unnamed Item ⋮ Average control of Markov decision processes with Feller transition probabilities and general action spaces ⋮ A linear programming formulation for constrained discounted continuous control for piecewise deterministic Markov processes ⋮ Sample-path optimality and variance-maximization for Markov decision processes ⋮ Constrained Markov decision processes in Borel spaces: from discounted to average optimality ⋮ Attack allocation on remote state estimation in multi-systems: structural results and asymptotic solution ⋮ Solutions of the average cost optimality equation for Markov decision processes with weakly continuous kernel: the fixed-point approach revisited ⋮ On the vanishing discount factor approach for Markov decision processes with weakly continuous transition probabilities ⋮ Another set of conditions for Markov decision processes with average sample-path costs ⋮ Nonzero-Sum Expected Average Discrete-Time Stochastic Games: The Case of Uncountable Spaces ⋮ Constrained semi-Markov decision processes with ratio and time expected average criteria in Polish spaces ⋮ The Vanishing Discount Approach for the Average Continuous Control of Piecewise Deterministic Markov Processes ⋮ Approximation of average cost Markov decision processes using empirical distributions and concentration inequalities ⋮ Zero-sum average cost semi-Markov games with weakly continuous transition probabilities and a minimax semi-Markov inventory problem

Cites Work

• Blackwell optimality in the class of stationary policies in Markov decision chains with a Borel state space and unbounded rewards
• Computable bounds for geometric convergence rates of Markov chains
• Adaptive Markov control processes
• Blackwell optimality in the class of all policies in Markov decision chains with a Borel state space and unbounded rewards
• Finite state Markovian decision processes
• STRONG AVERAGE OPTIMALITY FOR CONTROLLED NONHOMOGENEOUS MARKOV CHAINS^*
• Limiting Average Criteria For Nonstationary Markov Decision Processes
• Nonhomogeneous Markov Decision Processes with Borel State Space—The Average Criterion with Nonuniformly Bounded Rewards
• Average, Sensitive and Blackwell Optimal Policies in Denumerable Markov Decision Chains with Unbounded Rewards
• Control of Markov Chains with Long-Run Average Cost Criterion: The Dynamic Programming Equations
• Optimal Stationary Policies in General State Space Markov Decision Chains with Finite Action Sets
• Markov decision chains with unbounded costs and applications to the control of queues
• Contraction Conditions for Average and α-Discount Optimality in Countable State Markov Games with Unbounded Rewards
• Discrete-Time Controlled Markov Processes with Average Cost Criterion: A Survey
• Average cost Markov control processes with weighted norms: existence of canonical policies
• Arbitrary State Markovian Decision Processes
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item

This page was built for publication: Average optimality for Markov decision processes in borel spaces: a new condition and approach