Control of Markov Chains with Long-Run Average Cost Criterion: The Dynamic Programming Equations

DOI10.1137/0327034zbMath0668.60059OpenAlexW1976732220MaRDI QIDQ3821376

Vivek S. Borkar

Publication date: 1989

Published in: SIAM Journal on Control and Optimization (Search for Journal in Brave)

Full work available at URL: https://semanticscholar.org/paper/7f987f133d136e6a10e8e2dc36b0ae8ea896d2ce

zbMATH Keywords

dynamic programming equationsdiscrete time Markov chainsoptimal stable stationary strategy

Mathematics Subject Classification ID

Dynamic programming (90C39) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10) Optimal stochastic control (93E20)

Related Items (21)

On the average cost optimality equation and the structure of optimal policies for partially observable Markov decision processes ⋮ A note on the existence of optimal stationary policies for average Markov decision processes with countable states ⋮ Markov decision processes with multiple costs ⋮ Controlled diffusions with constraints ⋮ Recent results on conditions for the existence of average optimal stationary policies ⋮ The average cost of Markov chains subject to total variation distance uncertainty ⋮ Ergodic and adaptive control of nearest-neighbor motions ⋮ Average optimality in dynamic programming on Borel spaces -- unbounded costs and controls ⋮ On strong average optimality of Markov decision processes with unbounded costs ⋮ Comparing recent assumptions for the existence of average optimal stationary policies ⋮ Economic design of memory-type control charts: the fallacy of the formula proposed by Lorenzen and Vance (1986) ⋮ Unnamed Item ⋮ Controlled Markov chains with constraints. ⋮ Optimal Distributed Uplink Channel Allocation: A Constrained MDP Formulation ⋮ A survey of Markov decision models for control of networks of queues ⋮ Remarks on the existence of solutions to the average cost optimality equation in Markov decision processes ⋮ Infinite Horizon Average Cost Dynamic Programming Subject to Total Variation Distance Ambiguity ⋮ The Kumar-Becker-Lin scheme revisited ⋮ Average optimality for Markov decision processes in borel spaces: a new condition and approach ⋮ The convergence of value iteration in average cost Markov decision chains ⋮ Denumerable state stochastic games with limiting average payoff

This page was built for publication: Control of Markov Chains with Long-Run Average Cost Criterion: The Dynamic Programming Equations