Discrete Dynamic Programming with Sensitive Discount Optimality Criteria
A sequence of sensitive discount optimality criteria, unifying two criteria previously suggested by Blackwell, is introduced for both continuous and discrete time parameter finite Markov decision chains. Let \( V_{\rho}(\pi) \) denote the expected discounted reward under the policy \( \pi \) where \( \rho \) is the rate of interest. For each \( n=-1,0, \ldots \), a policy \( \pi^{*} \) is called \( n^{ \pm} \)optimal if \[ \liminf _{\rho \rightarrow 0 \pm}|\rho|^{-n_{\rho}}\left[V_{\rho}\left(\pi^{*}\right)-V_{\rho}(\pi)\right] \geq 0 \] for all \( \pi \), and \(\pm\) optimal if \( V_{\rho}\left(\pi^{*}\right)-V_{\rho}(\pi) \geq 0 \) for all \( \pi \) and all \( 0< \pm \rho<\rho^{*} \) for some \( \rho^{*}>0 \). When \( n \) is as large as the number of states, \( n^{ \pm} \)optimality is shown to be equivalent to \(\pm\) optimality. The recent policy improvement algorithm of Miller and the author for finding \(\pm\) optimal policies is refined and simplified to find \( n^{ \pm} \)optimal policies. The algorithm exploits the fact that for stationary policies the Laurent expansion about the origin of the return \( V_{p}(\cdot))\) has a simple form. Detailed properties of the coefficients of this expansion are exhibited. Resolvents of substochastic matrices play a central role in this analysis. The methods of successive approximation (Shapley) and policy improvement (Howard) are applied to finding a policy \( \pi \) maximizing \( V_{\rho}(\pi) \) for a fixed negative \( p \), provided the series defining \( V_{\rho}(\cdot) \) is absolutely convergent for all stationary policies.
- The variational calculus and approximation in policy space for Markovian decision processes
- Variational characterizations in Markov decision processes
- Markov decision processes with a minimum-variance criterion
- On fractional flow models and equivalent finite state processes
- A generalized inverse method for asymptotic linear programming
- Nonstationary Markov decision problems with converging parameters
- Conditions for characterizing the structure of optimal strategies in infinite-horizon dynamic programs
- Optimal control of Markov chains admitting strong and weak interactions
- Resolvent expansions of matrices and applications
- Denumerable semi-Markov decision chains with small interest rates
- Controlled jump processes
- Markov decision processes and strongly excessive functions
- Optimal threshold probability in undiscounted Markov decision processes with a target set.
- Singularly perturbed linear programs and Markov decision processes
- Gainfree Leontief substitution flow problems
- Are limits of -discounted optimal policies Blackwell optimal? A counterexample
- An asymptotic simplex method for singularly perturbed linear programs
- A canonical form for pencils of matrices with applications to asymptotic linear programs
- Approximating a diffusion by a finite-state hidden Markov model
- An efficient basis update for asymptotic linear programming
- Bias optimality and strong n (n= -1,0) discount optimality for Markov decision processes
- Error bounds for stochastic shortest path problems
- Reduction of total-cost and average-cost MDPs with weakly continuous transition probabilities to discounted mdps
- Recursive stochastic games with positive rewards
- Sample-path optimality and variance-maximization for Markov decision processes
- Blackwell optimality in the class of Markov policies for continuous-time controlled Markov chains
- Finite state multi-armed bandit problems: Sensitive-discount, average-reward and average-overtaking optimality
- Solving H-horizon, stationary Markov decision problems in time proportional to log (H)
- Markov branching decision chains with interest-rate-dependent rewards
- Computational methods for risk-averse undiscounted transient Markov models
- A two-queue polling model with priority on one queue and heavy-tailed on/off sources: a heavy-traffic limit
- On the reduction of total-cost and average-cost MDPs to discounted mdps
- Transient policies in discrete dynamic programming: Linear programming including suboptimality tests and additional constraints
- A Fixed Point Approach to Undiscounted Markov Renewal Programs
- A mixed value and policy iteration method for stochastic control with universally measurable policies
- Ergodic Control, Bias, and Sensitive Discount Optimality for Markov Diffusion Processes
- MARKOV DECISION PROCESSES
- Optimality equations and sensitive optimality in bounded Markov decision processes1
- The multi-armed bandit, with constraints
- Q-learning and policy iteration algorithms for stochastic shortest path problems
- An Efficient Factorization for the Group Inverse
- On the functional equations in undiscounted and sensitive discounted stochastic games
- First passage problems for nonstationary discrete-time stochastic control systems
- scientific article; zbMATH DE number 3711784 (Why is no real title available?)
- scientific article; zbMATH DE number 3731783 (Why is no real title available?)
- scientific article; zbMATH DE number 3743043 (Why is no real title available?)
- scientific article; zbMATH DE number 3469604 (Why is no real title available?)
- Survey of linear programming for standard and nonstandard Markovian control problems. Part I: Theory
- Simultaneous recurrent conditions on countable state Markov chains
- Note on discounted continuous-time Markov decision processes with a lower bounding function
- Blackwell optimal policies in a Markov decision process with a Borel state space
- Strong 0-discount optimal policies in a Markov decision process with a Borel state space
- Stability estimation of transient Markov decision processes
- Average cost optimality inequality for Markov decision processes with Borel spaces and universally measurable policies
- Maximum-stopping-value policies in finite Markov population decision chains
- Blackwell Optimality for Controlled Diffusion Processes
- Stochastic Abelian and Tauberian theorems
- Another Set of Conditions for Strongn(n = −1, 0) Discount Optimality in Markov Decision Processes
- An analysis of transient Markov decision processes
- Bounds on the fixed point of a monotone contraction operator
- Approximations for the distribution of perpetuities with small discount rates
- Stochastic control via direct comparison
- The relationships between discounted and average criteria of stochastic games with prospect theory
- Multiple-population discrete-time mean field games with discounted and total payoffs: the existence of equilibria
- Scaling limit of soliton lengths in a multicolor box-ball system
- Computation of optimal policies in discounted semi-Markov decision chains
- Strong n-discount and finite-horizon optimality for continuous-time Markov decision processes
- Continuous-time Markov decision processes with nth-bias optimality criteria
- Optimal threshold probability and expectation in semi-Markov decision processes
- A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder)
This page was built for publication: Discrete Dynamic Programming with Sensitive Discount Optimality Criteria
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5575236)