scientific article; zbMATH DE number 3320878
action spaceBayesian approachBellman's principle of optimalityBorel subsets of complete separable metric spacesdynamic programming under uncertaintyMarkovian stationary modelsmodels with incomplete information and randomized plansnon-stationary dynamic programming under riskoptimality criteriaoptimality equationstate space
The treatise is divided in two chapters. The first chapter treats models with countable state space and arbitrary action space. The problem of non-stationary dynamic programming under risk is formulated as an appropriate tupel \((A, S, (D_n), (p_n), (r_n))\), where \(S\) is a countable state space, \(A\) is an action space, \(D_n(h)\) is the set of actions available to us if at time \(n\) the history \(h=(s_1, a_1, \ldots, s_n) \in (S \times A \times S \cdots \times S)\) has occurred, \(p_0 (s_1)\) is the initial distribution, \(p_n(h, a, s)\) for \(n \ge 1\) is the probability to be in state \(\underline{s}\) at time \((n+1)\), if at time \(\underline{n}\) the history \(h\) has occurred and if we take action \(\underline{a}\), finally \(r_n(h, a)\) is the reward given to us at time \((n+1)\). Time is discrete in the whole book. Further it is assumed that either all reward functions are essentially positive or essentially negative; the notion of \(\bar{p}\)-optimality is introduced. The relation between Bellman's principle of optimality and the optimality equation derived in the book, is studied. Further a number of optimality criteria are described and the important concept of sufficient statistics is defined. The author then confines the treatment to Markovian stationary models and derives some results on so-called models with incomplete information and randomized plans. At the end of Chapter I the analysis of dynamic programming under uncertainty by means of the Bayesian approach is brought. In Chapter II the author investigates models, where the state space and action space are Borel subsets of complete separable metric spaces. The results analogous to those derived in Chapter I are obtained for such type of models. The arguments already used in Chapter I are not repeated and the author concentrates on the measure-theoretic aspects of the problem. The whole Chapter II assumes the reader has a good background in measure theory and topology. To those who have only elementary knowledge in these fields three Appendices are devoted.
- Dynamic mean-risk optimization in a binomial model
- On a stopping rule for a class of sequential decision problems
- Stochastic control theory and operational research
- Optimal research and development expenditures under an incremental tax incentive scheme
- Nonstationary value-iteration and adaptive control of discounted semi- Markov processes
- Stationary policies and Markov policies in Borel dynamic programming
- Semicontinuous nonstationary stochastic games
- On -optimal continuous selectors and their application in discounted dynamic programming
- A polynomial time bound for Howard's policy improvement algorithm
- Finite-state approximations for denumerable multidimensional state discounted Markov decision processes
- Fixed point theorems for discounted finite Markov decision processes
- Utility, probabilistic constraints, mean and variance of discounted rewards in Markov decision processes
- Sufficient conditions for optimality of a (z,c^ -,c^ +)-sampling plan in multistage Bayesian acceptance sampling
- A unified approach to adaptive control of average reward Markov decision processes
- Adaptive policies for discrete-time stochastic control systems with unknown disturbance distribution
- Continuous dependence of stochastic control models on the noise distribution
- Necessary and sufficient conditions for a bounded solution to the optimality equation in average reward Markov decision chains
- The Bellman's principle of optimality in the discounted dynamic programming
- On compactness of the space of policies in stochastic dynamic programming
- Nonparametric adaptive control of discrete-time partially observable stochastic systems
- Robustness inequality for Markov control processes with unbounded costs
- A pause control approach to the value iteration scheme in average Markov decision processes
- A note on the convergence rate of the value iteration scheme in controlled Markov chains
- \(C^3\) modeling with symmetrical rationality
- Markov renewal decision processes with finite horizon
- A remark on the connections between coding and dynamic programming
- Conditions for characterizing the structure of optimal strategies in infinite-horizon dynamic programs
- A natural extension of the MacQueen extrapolation
- Some comments on preference order dynamic programming models
- A dual approach to Bayesian inference and adaptive control
- Optimal policies in multiproduct inventory models
- On two-state quality control under Markovian deterioration
- Recent results on conditions for the existence of average optimal stationary policies
- Nonparametric estimation and adaptive control in a class of finite Markov decision chains
- Average cost Markov decision processes: Optimality conditions
- Estimation and control in multichain processes
- Structured policies in the sequential design of experiments
- Optimal dynamic load distribution in a class of flow-type flexible manufacturing systems
- Optimal stationary policies in the vector-valued Markov decision process
- Equivalence of Lyapunov stability criteria in a class of Markov decision processes
- Existence of optimal stationary policies in average reward Markov decision processes with a recurrent state
- Evolution and market behavior
- Markov-Entscheidungs-Prozesse mit abhängigen Aktionen für optimale Reparaturmaßnahmen bei unvollständiger Information. (Markov decision processes with dependent actions for optimal repair policies under incomplete information)
- A limited order capacity stochastic inventory model with a fixed cost for order: The discounted case
- Controlled jump processes
- On dynamic programming: Compactness of the space of policies
- On stopped decision processes with discrete time parameter
- Estimates for finite-stage dynamic programs
- Finite-state, discrete-time optimization with randomly varying observation quality
- Dynamic programming of expectation and variance
- A selection theorem for optimization problems
- Stochastische dynamische Optimierung als Spezialfall linearer Optimierung in halbgeordneten Vektorräumen
- On a representation of measurable automaton transformations by stochastic automata
- Discounted, positive, and noncooperative stochastic games
- Measurable selection theorems for optimization problems
- On some aspects in stochastic dynamic programming with terminal region
- Dynamic programming and principles of optimality
- Conditional decision processes with recursive function
- Bounds for the quality and the number of steps in Bellman's value iteration algorithm
- On Nash equilibrium solutions in nonzero-sum stochastic games with complete information
- Approximation of average cost optimal policies for general Markov decision processes with unbounded costs
- Randomization and simplification in dynamic decision-making.
- Constrained denumerable state non-stationary MDPs with expected total reward criterion
- Optimal strategies for an inventory system with cost functions of general form
- Risk measurement and risk-averse control of partially observable discrete-time Markov systems
- Portfolio optimization under dynamic risk constraints: continuous vs. discrete time trading
- Zum Problem des zweiarmigen Bernoulli-Banditen mit einer bekannten Erfolgswahrscheinlichkeit und unendlich vielen Spielen
- Monotonicity and the principle of optimality
- Controlled Markov set-chains under average criteria
- Minimax control for discrete-time time-varying stochastic systems
- Vector-valued Markov decision processes and the systems of linear inequalities
- Value iteration in average cost Markov control processes on Borel spaces
- The transformation method for continuous-time Markov decision processes
- Markov-achievable payoffs for finite-horizon decision models.
- Stability estimation of some Markov controlled processes
- A consumption and investment problem via a Markov decision processes approach with random horizon
- Discrete-time ergodic mean-field games with average reward on compact spaces
- On the expected total reward with unbounded returns for Markov decision processes
- First-order sensitivity of the optimal value in a Markov decision model with respect to deviations in the transition probability function
- Stochastic dynamic programming with non-linear discounting
- Minimizing spectral risk measures applied to Markov decision processes
- Recursive adaptive control of Markov decision processes with the average reward criterion
- Average cost optimal policies for Markov control processes with Borel state space and unbounded costs
- Learning and self-confirming long-run biases
- Semi-Markov decision processes with variance minimization criterion
- Dynamic CVAR with multi-period risk problems
- Markov control processes with randomized discounted cost
- Dynamic risk measures under model uncertainty
- Credibilistic Markov decision processes: The average case
- Instationäre dynamische Optimierung bei schwachen Voraussetzungen über die Gewinnfunktionen
- Increasing Lipschitz continuous maximizers of some dynamic programs
- Recurrence conditions for Markov decision processes with Borel state space: A survey
- Density estimation and adaptive control of Markov processes: Average and discounted criteria
- Asymptotic optimality and rates of convergence of quantized stationary policies in continuous-time Markov decision processes
- STRONG AVERAGE OPTIMALITY FOR CONTROLLED NONHOMOGENEOUS MARKOV CHAINS*
- The bellman equation for vector-valued semi-markovian dyanmic programiing
- Partially observable total-cost Markov decision processes with weakly continuous transition probabilities
- Characterizations of optimal policies in a general stopping problem and stability estimating
- Constrained Markov control processes with randomized discounted cost criteria: infinite linear programming approach
- Markov decision processes with iterated coherent risk measures
This page was built for publication:
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5599448)