Linear programming approximations for Markov control processes in metric spaces

DOI10.1023/A:1005826226226MaRDI QIDQ1387656zbMATH OpenOpenAlexFDO

Authors Onésimo Hernández-Lerma, Jean B. Lasserre

Publication date 30 May 1999

Published in Acta Applicandae Mathematicae (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1023/a:1005826226226

zbMATH Keywords

discrete-time Markov control processeslinear programming

Mathematics Subject Classification ID

Linear programming (90C05) Markov and semi-Markov decision processes (90C40) Optimal stochastic control (93E20)

Review text

The authors develop linear programming techniques (LP) for discrete-time Markov control processes (MCP) in an uncountable metric space, particularly suitable for computational purposes. First, the authors introduce the associated LP equivalent to the MCP in the sense that \(\rho^*=\min\mathbb{P}\), where \(\rho^*\) and \(\min\mathbb{P}\) are the optimum values of MCP and LP respectively [cf. the authors' book, Discrete-time Markov control processes: Basic optimality criteria (1995; Zbl 0840.93001)]. Then, using a suitable aggregation-relaxation procedure to approximate \(\mathbb{P}\) by LPs with a finite number of decision variables, it is shown that under mild assumptions these procedures converge to \(\min \mathbb{P}\). The basic difference between the schemes of the authors and those of other researchers is that the authors use approximations in weak (or weak*) topologies rather than in normed topologies.

Recommendations

• Finite linear programming approximations of constrained discounted Markov decision processes
• Infinite Linear Programming and Multichain Markov Control Processes in Uncountable Spaces
• Linear Programming and Average Optimality of Markov Control Processes on Borel Spaces—Unbounded Costs
• Approximation of Infinite-Dimensional Linear Programming Problems which Arise in Stochastic Control

Cited in

(5)

• Linearization techniques for controlled piecewise deterministic Markov processes; application to Zubov's method
• Constrained Markov control processes with randomized discounted cost criteria: infinite linear programming approach
• A Direct Approach to the Solution of Optimal Multiple-Stopping Problems
• Survey of linear programming for standard and nonstandard Markovian control problems. Part II: Applications
• Approximations of Countably Infinite Linear Programs over Bounded Measure Spaces

This page was built for publication: Linear programming approximations for Markov control processes in metric spaces

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1387656)