Technical Note—An Equivalence Between Continuous and Discrete Time Markov Decision Processes
From MaRDI portal
Publication:3048616
DOI10.1287/opre.27.3.616zbMath0413.90079OpenAlexW1964680067WikidataQ56907741 ScholiaQ56907741MaRDI QIDQ3048616
Publication date: 1979
Published in: Operations Research (Search for Journal in Brave)
Full work available at URL: https://doi.org/10.1287/opre.27.3.616
infinite horizondynamic programmingMarkov chainscontrol of queuescountable state spacecontinuous time Markov decision processdiscrete time Markov decision process
Minimax problems in mathematical programming (90C47) Discrete-time control/observation systems (93C55) Dynamic programming (90C39) Applications of Markov renewal processes (reliability, queueing networks, etc.) (60K20) Model systems in control theory (93C99)
Related Items (64)
A new condition for the existence of optimal stationary policies in average cost Markov decision processes ⋮ On the control of a compound immigration process through total catastrophes ⋮ Kolmogorov's Equations for Jump Markov Processes and Their Applications to Control Problems ⋮ Optimal control of a multiclass queueing system when customers can change types ⋮ Assigning multiple job types to parallel specialized servers ⋮ Control and scheduling in a two-station queueing network: Optimal policies and heuristics ⋮ OPTIMAL ADMISSION AND ROUTING WITH CONGESTION-SENSITIVE CUSTOMER CLASSES ⋮ On arrival driven queueing models: Admission control, traffic policing, abandonments, and correlated arrivals ⋮ On the Control of a Queueing System with Aging State Information ⋮ Simultaneous optimization of flow control and scheduling in a single server queue with two job classes ⋮ Simultaneous optimization of flow-control and scheduling in a single server queue with two job classes: Numerical results and approximation ⋮ Coordinated production and inspection in a tandem system ⋮ PH-graphs for analyzing shortest path problems with correlated traveling times ⋮ Optimal decisions for continuous time Markov decision processes over finite planning horizons ⋮ An assignment problem for a parallel queueing system with two heterogeneous servers ⋮ Queueing Network Controls via Deep Reinforcement Learning ⋮ A useful technique for piecewise deterministic Markov decision processes ⋮ A multi-objective approach for PH-graphs with applications to stochastic shortest paths ⋮ Optimal control of polling models for transportation applications ⋮ Numerical analysis of continuous time Markov decision processes over finite horizons ⋮ A queueing model for customer rescheduling and no-shows in service systems ⋮ Assigning a single server to inhomogeneous queues with switching costs ⋮ Scheduling Markovian PERT networks to maximize the net present value: new results ⋮ Two‐class constrained optimization with applications to queueing control ⋮ The transformation method for continuous-time Markov decision processes ⋮ Dynamic control of a single-server system with abandonments ⋮ Discrete-time switching control in random walks ⋮ Optimality of the fastest available server policy ⋮ On theory and algorithms for Markov decision problems with the total reward criterion ⋮ Optimal control of a queueing system with an exponential and an Erlangian server and renewal input stream ⋮ Strong polynomiality of the Gass-Saaty shadow-vertex pivoting rule for controlled random walks ⋮ Action-dependent stopping times and Markov decision process with unbounded rewards ⋮ A conservative index heuristic for routing problems with multiple heterogeneous service facilities ⋮ On structural properties of the value function for an unbounded jump Markov process with an application to a processor sharing retrial queue ⋮ Joint dynamic pricing and capacity control for hotels and rentals with advanced demand information ⋮ TWO-CLASS ROUTING WITH ADMISSION CONTROL AND STRICT PRIORITIES ⋮ Computation of weighted sums of rewards for concurrent MDPs ⋮ Some models of queueing control with switchover ⋮ Dynamic vs static pricing in a make-to-stock queue with partially controlled production ⋮ Stability criteria for multi-class queueing networks with re-entrant lines ⋮ Monotonicity properties of cost functions in queueing networks ⋮ Optimal use of excess capacity in two interconnected queues ⋮ Approximate uniformization for continuous-time Markov chains with an application to performability analysis ⋮ Customizing exponential semi-Markov decision processes under the discounted cost criterion ⋮ Light robustness in the optimization of Markov decision processes with uncertain parameters ⋮ Continuous time markov decision processes with nonuniformly bounded transition rate: expected total rewards ⋮ Computation of the optimal policy for the control of a compound immigration process through total catastrophes ⋮ On the control of a truncated general immigration process through the introduction of a predator ⋮ Stability criteria for controlled queueing systems ⋮ A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder) ⋮ Optimal server assignment in a two-stage tandem queueing system ⋮ Protocol invariance and the timing of decisions in dynamic games ⋮ Optimal control and performance analysis of anMX/M/1queue with batches of negative customers ⋮ OPTIMAL PRICING AND PRODUCTION POLICIES OF A MAKE-TO-STOCK SYSTEM WITH FLUCTUATING DEMAND ⋮ Scheduling projects with stochastic activity duration to maximize expected net present value ⋮ A novel use of value iteration for deriving bounds for threshold and switching curve optimal policies ⋮ Dynamic dispatching and preventive maintenance for parallel machines with dispatching-dependent deterioration ⋮ \(K\) competing queues with customer abandonment: optimality of a generalised \(c \mu \)-rule by the smoothed rate truncation method ⋮ Concurrent MDPs with Finite Markovian Policies ⋮ Control of arrivals to two queues in series ⋮ An optimal maintenance policy for a server with decreasing arrival rate. ⋮ Look-back policies for two-stage, pull-type production/inventory systems ⋮ Lock and no-lock mortgage plans: Is it only a matter of risk shifting? ⋮ Optimal dynamic routing in Markov queueing networks
This page was built for publication: Technical Note—An Equivalence Between Continuous and Discrete Time Markov Decision Processes
