Markov control process with the expected total cost criterion: Optimality, stability, and transient models

DOI10.1023/A:1006368714127MaRDI QIDQ1973302zbMATH OpenOpenAlexFDO

Authors Guadalupe Carrasco, Rubén Pérez-Hernández, Onésimo Hernández-Lerma

Publication date 29 August 2000

Published in Acta Applicandae Mathematicae (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1023/a:1006368714127

zbMATH Keywords

discrete-time Markov control processesdynamic programmingexpected total costLagrange stabilitypolicy iterationstability with probability onetransient control models

Mathematics Subject Classification ID

Markov and semi-Markov decision processes (90C40) Optimal stochastic control (93E20)

Review text

The authors study discrete-time Markov Control Processes (MCPs) on Borel spaces under the Expected Total Cost (ETC) criterion \[ V(\pi, x)= E^\pi_x\Biggl[ \sum^\infty_{t= 0} c(x_ t,a_t)\Biggr], \] where \(c(x_t, a_t)\) is the cost-per-stage function and is possibly unbounded [for the basic concepts and notations of the MCPs, cf. \textit{O. Hernández-Lerma} and \textit{J. B. Lasserre}, Discrete-time Markov control processes: Basic optimality criteria, Springer-Verlag, New York (1995; Zbl 0840.93001)]. A lot of optimality questions are answered affirmatively here. Conditions for a control policy to be ETC-optimal and conditions for the ETC-value function to be a solution to the dynamic programming equation are well provided. It is also shown that the finiteness of the ETC function may lead to two kinds of stability: Lagrange stability and stability with probability one. In addition, transient control models [cf. \textit{S. R. Pliska}, Dynamic programming and its applications, Proc. Int. Conf., Vancouver 1977, 335-349 (1978; Zbl 0458.90082)] are fully analyzed. In fact, with the authors' new results, the paper provides a fairly complete, up-dated, survey-like presentation of the ETC criterion for MCPs.

Cited in

(7)

• A finite exact algorithm to solve a dice game
• Markov decision processes with iterated coherent risk measures
• Robust shortest path planning and semicontractive dynamic programming
• scientific article; zbMATH DE number 179178 (Why is no real title available?)
• scientific article; zbMATH DE number 1325008 (Why is no real title available?)
• Stability estimation of transient Markov decision processes
• An analysis of transient Markov decision processes

This page was built for publication: Markov control process with the expected total cost criterion: Optimality, stability, and transient models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1973302)