Dynamic optimization. Deterministic and stochastic models

DOI10.1007/978-3-319-48814-1MaRDI QIDQ332155zbMATH OpenOpenAlexFDO

Authors K. Hinderer, Ulrich Rieder, Michael Stieglitz

Publication date 27 October 2016

Published in Universitext (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1007/978-3-319-48814-1

dynamic optimization dynamic programming Markovian decision process stochastic control stochastic dynamic program

Discrete-time Markov processes on general state spaces (60J05) Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) (60J20) Introductory exposition (textbooks, tutorial papers, etc.) pertaining to operations research and mathematical programming (90-01) Research exposition (monographs, survey articles) pertaining to operations research and mathematical programming (90-02) Deterministic network models in operations research (90B10) Dynamic programming (90C39) Markov and semi-Markov decision processes (90C40) Optimal stochastic control (93E20)

Review text

Part I deals with deterministic dynamic optimization models describing the control of discrete-time systems. The problems are defined by the final sequences of states, actions, and by the transition function moving the system into the new state with the purpose of maximizing the sum of discounted reward functions or minimizing the sum of discounted cost functions. The authors prove the optimality criteria and develop procedures to solve the problems. In Chapter 5 they consider absorbing dynamic programs and solve the problem of searching cost-minimal subpaths in acyclic network. Special chapters are concerned with the structure of the value functions and of maximizers. Finally, the authors present the general theory for the case when the number of stages goes to infinity and derive asymptotically optimal decision rules. Part II is devoted to discrete-time stochastic control models. In Chapters 11, 12 the authors define control models with independent identical disturbances and Markovian decision processes with finite transition law. Problems of maximizing expected rewards and minimizing expected costs are considered. The structural properties of the optimal solution are studied just as the asymptotic behaviour of the solution for infinite set of states. Asymptotically optimal decision rules are deduced. The optimality criteria are proved and solution methods are developed. In Chapter 16 the authors investigate Markovian decision processes with arbitrary transition law, where states and actions are measurable spaces or separable metric spaces. The authors derive conditions when the problem can be solved and when the optimal policy exists. Special chapters are devoted to structural properties of the value functions. Part III ``Generalization of Markov Decision Processes is devoted to Markovian decision processes with disturbances and considers Markov renewal programs, Bayesian control models, and partially observable models. The book comprises a lot of examples, problems for readers, and supplements with additional comments for the advanced reader and with bibliographic notes.

Recommendations

Cited in

(31)

This page was built for publication: Dynamic optimization. Deterministic and stochastic models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q332155)