Sample path optimality for a Markov optimization problem

DOI10.1016/J.SPA.2004.12.005MaRDI QIDQ2485850zbMATH OpenOpenAlexFDO

Authors Fern Y. Hunt

Publication date 5 August 2005

Published in Stochastic Processes and their Applications (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1016/j.spa.2004.12.005

zbMATH Keywords

Azuma's inequalityMarkov decision processStochastic control

Mathematics Subject Classification ID

Markov chains (discrete-time Markov processes on discrete state spaces) (60J10) Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) (60J20) Markov and semi-Markov decision processes (90C40) Optimal stochastic control (93E20)

Review text

A controlled Markov chain with long-term expected average cost criterion is considered. Sufficient conditions are given under which an optimal stationary policy is also asymptotically optimal with respect to the sample path cost criterion.

Recommendations

• Sample-path average optimality for Markov control processes
• Sample-Path Optimality and Variance-Minimization of Average Cost Markov Control Processes
• Markov Decision Processes with Sample Path Constraints: The Communicating Case
• Sample path average optimality of Markov control processes with strictly unbounded cost
• Sample-Path Optimal Stationary Policies in Stable Markov Decision Chains with the Average Reward Criterion

Cites work

• Hoeffding's inequality for uniformly ergodic Markov chains
• scientific article; zbMATH DE number 4218449 (Why is no real title available?)
• scientific article; zbMATH DE number 1348599 (Why is no real title available?)
• scientific article; zbMATH DE number 964350 (Why is no real title available?)
• scientific article; zbMATH DE number 3085434 (Why is no real title available?)
• Markov Chains
• Markov chains and stochastic stability
• Markov Decision Problems and State-Action Frequencies
• On asymptotic optimality in probability and almost surely in dynamic control
• On Optimality in Probability and Almost Surely for Processes with a Communication Property. I. The Discrete Time Case
• Pathwise optimality in stochastic control

Cited in

(9)

• Single sample path-based optimization of Markov chains
• Reading policies for joins: an asymptotic analysis
• Sample-Path Optimal Stationary Policies in Stable Markov Decision Chains with the Average Reward Criterion
• scientific article; zbMATH DE number 3901881 (Why is no real title available?)
• Sample-path average optimality for Markov control processes
• Sample-path optimality in average Markov decision chains under a double Lyapunov function condition
• Denumerable controlled Markov chains with average reward criterion: Sample path optimality
• A counterexample on sample-path optimality in stable Markov decision chains with the average reward criterion
• Pathwise Optimality for Benchmark Tracking

This page was built for publication: Sample path optimality for a Markov optimization problem

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2485850)