The value iteration method for countable state Markov decision processes

DOI10.1016/S0167-6377(99)00015-2MaRDI QIDQ1306452zbMATH OpenWikidataFDO

Authors Yossi Aviv, Awi Federgruen

Publication date 9 February 2001

Published in Operations Research Letters (Search for Journal in Brave)

zbMATH Keywords

Markov decision processesunbounded costsoptimal policycountable state spacevalue-iteration method

Mathematics Subject Classification ID

Markov and semi-Markov decision processes (90C40)

Recommendations

• Value iteration in countable state average cost Markov decision processes with unbounded costs
• Value iteration and approximately optimal stationary policies in finite-state average Markov decision chains
• Value Iteration in a Class of Communicating Markov Decision Chains with the Average Cost Criterion
• Uniform convergence of value iteration policies for discounted Markov decision processes

Cites work

• scientific article; zbMATH DE number 4013703 (Why is no real title available?)
• scientific article; zbMATH DE number 3151099 (Why is no real title available?)
• scientific article; zbMATH DE number 3906232 (Why is no real title available?)
• scientific article; zbMATH DE number 3718880 (Why is no real title available?)
• scientific article; zbMATH DE number 3718883 (Why is no real title available?)
• scientific article; zbMATH DE number 700091 (Why is no real title available?)
• An Inventory Model with Limited Production Capacity and Uncertain Demands I. The Average-Cost Criterion
• Average Cost Optimal Stationary Policies in Infinite State Markov Decision Processes with Unbounded Costs
• Bounds and Asymptotics for Planning Critical Safety Stocks
• Comparing recent assumptions for the existence of average optimal stationary policies
• Computing the optimal policy for capacitated inventory models
• Denumerable Undiscounted Semi-Markov Decision Processes with Unbounded Rewards
• On strong average optimality of Markov decision processes with unbounded costs
• On the Second Optimality Equation for Semi-Markov Decision Models
• Semi-Markov Decision Processes with Unbounded Rewards
• Sensitivity Analysis for Base-Stock Levels in Multiechelon Production-Inventory Systems
• Stochastic Inventory Models with Limited Production Capacity and Periodically Varying Parameters
• Stochastic optimal control. The discrete time case
• The asymptotic behaviour of the minimal total expected cost for the denumerable state Markov decision model
• The convergence of value iteration in average cost Markov decision chains
• Value iteration in countable state average cost Markov decision processes with unbounded costs
• X-Y Band and Modified (s, S) Policy

Cited in

(22)

• Deciding the Value 1 Problem for $\sharp$ -acyclic Partially Observable Markov Decision Processes
• A continuous review model with general shelf age and delay-dependent inventory costs
• Analysis of the optimal resource allocation for a tandem queueing system
• Maximizing the probability of visiting a set infinitely often for a countable state space Markov decision process
• Optimal control of a two-server heterogeneous queueing system with breakdowns and constant retrials
• Algorithms for Countable State Markov Decision Models with an Absorbing Set
• scientific article; zbMATH DE number 3980960 (Why is no real title available?)
• Structural properties of the optimal resource allocation policy for single-queue systems
• Heuristic solution for the optimal thresholds in a controllable multi-server heterogeneous queueing system without preemption
• Illustrated review of convergence conditions of the value iteration algorithm and the rolling horizon procedure for average-cost MDPs
• Stochastic Inventory Models with Limited Production Capacity and Periodically Varying Parameters
• Technical note -- cyclic variables and Markov decision processes
• Value set iteration for Markov decision processes
• Optimal threshold control of a retrial queueing system with finite buffer
• On convergence of value iteration for a class of total cost Markov decision processes
• Uniform convergence of value iteration policies for discounted Markov decision processes
• An optimal control approach to day-to-day congestion pricing for stochastic transportation networks
• scientific article; zbMATH DE number 3916050 (Why is no real title available?)
• The Value Iteration Algorithm in Risk-Sensitive Average Markov Decision Chains with Finite State Space
• The convergence of value iteration in discounted Markov decision processes
• Isotone policies for the value iteration method for Markov decision processes
• An analysis of transient Markov decision processes

This page was built for publication: The value iteration method for countable state Markov decision processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1306452)