scientific article; zbMATH DE number 420890
About Markov Decision Processes (MDP), many books and an immense number of papers have been written. The present book is a remarkable book of a remarkable author. As the author remarks in his foreword, the book leans on three monographs of R. A. Howard, J. van der Wal and L. C. M. Kallenberg, resp., which also, in this order, determine the content of the first four chapters. Chapter 1 nicely makes the reader acquainted with the \(z\)-transform, a method of generating functions. In Chapter 2, the framework of MDP using sequences of random variables (state, action, reward) is built. There are presented the problem classes ``Finite Horizon, ``Discounted, ``Average and ``Absorbed with all of their particularities. Narrowly connected with it is Chapter 3, ``Algorithms. Here, for each problem class, the known methods of value and policy iteration are presented, their advantages and disadvantages are weighted one against the other, and refined techniques (elimination of suboptimal strategies) are given. In Chapter 4, for all problem classes, a transformation into a linear programming problem follows, shortly, because Kallenberg had it done already more comprehensively. Chapter 5 introduces into semi-MDP whereby, for simplicity, sojourn times are assumed as discretely distributed. The often more realistic case that the current state cannot be observed itself but only by an indicator (partial observation) and also the case when transition law and reward still depend on an unknown parameter (but with known distribution) being estimated during the observation (adaptive MDP) has been considered in Chapter 6. Chapter 7 simply named ``Further Aspects of MDP reflects original ideas of the author published in his numerous earlier papers. This concerns structured policies, approximation modelling, sensitivity analysis, multiobjective MDP, mean-variance criteria as well as Markov games. After that already earlier applications are named, Chapter 8 finally, at a scarce amount, describes real-life examples which can be formulated as MDP. The book is compact, written naturally and in an ingenious style. It has an entirely proper terminology (Result instead of Proposition) and symbolic (limit instead of lim). It refers to results of earlier research (especially also to the author's book ``Finite Dynamic Programming (1978; Zbl 0429.90043) without extending the list of references (79 items) too much. It is a compendium for students of many faculties as mathematics, statistics, operations research, systems and industrial engineering as well as management science but also lectures and researchers in those field would profit from its use. It is also suited for engineers, because the author often skips mathematical difficulties (measurability, Markov property). For all but one chapter, there exist exercises with solutions, carefully elaborated. Only a few critical remarks may be given: In formulae with ratios, the fraction bars are put in a rather discordant way \((/,-)\). The notation of a conditional random variable in formula (2.24) seems doubtful. Also the book is not entirely free of misprints.
- Optimal infinite-horizon multicriteria feedback control of stationary systems with minimax objectives and bounded disturbances
- Bounded-parameter Markov decision processes
- Illustrated review of convergence conditions of the value iteration algorithm and the rolling horizon procedure for average-cost MDPs
- Simulation-based algorithms for Markov decision processes
- Approximate dynamic programming for stochastic \(N\)-stage optimization with application to optimal consumption under uncertainty
- Competing Markov decision processes
- An axiomatic approach to Markov decision processes
- Sensitivity analysis in Markov decision processes with uncertain reward parameters
- scientific article; zbMATH DE number 3854800 (Why is no real title available?)
- scientific article; zbMATH DE number 4154239 (Why is no real title available?)
- Optimal selection policies for a sequence of candidate drugs
- Derman's book as inspiration: some results on LP for MDPs
- scientific article; zbMATH DE number 45269 (Why is no real title available?)
- scientific article; zbMATH DE number 700091 (Why is no real title available?)
- Markov Decision Processes with Imprecise Transition Probabilities
- Competitive Markov decision processes
- scientific article; zbMATH DE number 2062343 (Why is no real title available?)
- scientific article; zbMATH DE number 6932166 (Why is no real title available?)
- scientific article; zbMATH DE number 7014212 (Why is no real title available?)
- Quantile Markov Decision Processes
- Robust Markov Decision Processes
- Multitime scale markov decision processes
- Multi-Actor Markov Decision Processes
- scientific article; zbMATH DE number 2189770 (Why is no real title available?)
- scientific article; zbMATH DE number 5658801 (Why is no real title available?)
- Markov decision processes
- Markov decision processes
- Exact decomposition approaches for Markov decision processes: a survey
- Risk-averse dynamic programming for Markov decision processes
- Markov decision processes with applications to finance.
- Quantile Fourier regressions for decision making under uncertainty
- Evolution time of stochastic systems with multiple final sequences of states
- Simulation-based algorithms for Markov decision processes.
- Integrating stochastic reasoning into Event-B development
This page was built for publication:
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3135096)