Technical Note—An Equivalence Between Continuous and Discrete Time Markov Decision Processes
From MaRDI portal
Publication:3048616
DOI10.1287/OPRE.27.3.616zbMATH Open0413.90079DBLPjournals/ior/Serfozo79OpenAlexW1964680067WikidataQ56907741 ScholiaQ56907741MaRDI QIDQ3048616FDOQ3048616
Authors: Richard F. Serfozo 
Publication date: 1979
Published in: Operations Research (Search for Journal in Brave)
Full work available at URL: https://doi.org/10.1287/opre.27.3.616
dynamic programmingMarkov chainsinfinite horizoncontrol of queuescountable state spacecontinuous time Markov decision processdiscrete time Markov decision process
Dynamic programming (90C39) Minimax problems in mathematical programming (90C47) Applications of Markov renewal processes (reliability, queueing networks, etc.) (60K20) Discrete-time control/observation systems (93C55) Model systems in control theory (93C99)
Cited In (65)
- Stability criteria for controlled queueing systems
- An optimal maintenance policy for a server with decreasing arrival rate.
- Control of arrivals to two queues in series
- OPTIMAL PRICING AND PRODUCTION POLICIES OF A MAKE-TO-STOCK SYSTEM WITH FLUCTUATING DEMAND
- Joint dynamic pricing and capacity control for hotels and rentals with advanced demand information
- A new condition for the existence of optimal stationary policies in average cost Markov decision processes
- Optimal use of excess capacity in two interconnected queues
- Computation of weighted sums of rewards for concurrent MDPs
- Assigning a single server to inhomogeneous queues with switching costs
- Dynamic dispatching and preventive maintenance for parallel machines with dispatching-dependent deterioration
- Protocol invariance and the timing of decisions in dynamic games
- Numerical analysis of continuous time Markov decision processes over finite horizons
- Optimal control of a queueing system with an exponential and an Erlangian server and renewal input stream
- Optimal control of a multiclass queueing system when customers can change types
- On the control of a compound immigration process through total catastrophes
- A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder)
- On structural properties of the value function for an unbounded jump Markov process with an application to a processor sharing retrial queue
- Assigning multiple job types to parallel specialized servers
- Stability criteria for multi-class queueing networks with re-entrant lines
- Dynamic control of a single-server system with abandonments
- Customizing exponential semi-Markov decision processes under the discounted cost criterion
- On arrival driven queueing models: Admission control, traffic policing, abandonments, and correlated arrivals
- Computation of the optimal policy for the control of a compound immigration process through total catastrophes
- Optimal control and performance analysis of anMX/M/1queue with batches of negative customers
- A conservative index heuristic for routing problems with multiple heterogeneous service facilities
- Simultaneous optimization of flow control and scheduling in a single server queue with two job classes
- Control and scheduling in a two-station queueing network: Optimal policies and heuristics
- Queueing network controls via deep reinforcement learning
- On theory and algorithms for Markov decision problems with the total reward criterion
- Action-dependent stopping times and Markov decision process with unbounded rewards
- Optimal decisions for continuous time Markov decision processes over finite planning horizons
- Scheduling projects with stochastic activity duration to maximize expected net present value
- Optimal server assignment in a two-stage tandem queueing system
- On the control of a truncated general immigration process through the introduction of a predator
- Approximate uniformization for continuous-time Markov chains with an application to performability analysis
- Look-back policies for two-stage, pull-type production/inventory systems
- The transformation method for continuous-time Markov decision processes
- A multi-objective approach for PH-graphs with applications to stochastic shortest paths
- PH-graphs for analyzing shortest path problems with correlated traveling times
- Discrete-time switching control in random walks
- Optimal dynamic routing in Markov queueing networks
- Some models of queueing control with switchover
- Scheduling Markovian PERT networks to maximize the net present value: new results
- Optimal control of polling models for transportation applications
- An assignment problem for a parallel queueing system with two heterogeneous servers
- Monotonicity properties of cost functions in queueing networks
- Two‐class constrained optimization with applications to queueing control
- Strong polynomiality of the Gass-Saaty shadow-vertex pivoting rule for controlled random walks
- Optimality of the fastest available server policy
- \(K\) competing queues with customer abandonment: optimality of a generalised \(c \mu \)-rule by the smoothed rate truncation method
- Dynamic vs static pricing in a make-to-stock queue with partially controlled production
- Coordinated production and inspection in a tandem system
- Light robustness in the optimization of Markov decision processes with uncertain parameters
- Lock and no-lock mortgage plans: Is it only a matter of risk shifting?
- Continuous time markov decision processes with nonuniformly bounded transition rate: expected total rewards
- Kolmogorov's equations for jump Markov processes and their applications to control problems
- Optimal admission and routing with congestion-sensitive customer classes
- Logarithmic regret bounds for continuous-time average-reward Markov decision processes
- A novel use of value iteration for deriving bounds for threshold and switching curve optimal policies
- Simultaneous optimization of flow-control and scheduling in a single server queue with two job classes: Numerical results and approximation
- Concurrent MDPs with Finite Markovian Policies
- A useful technique for piecewise deterministic Markov decision processes
- A queueing model for customer rescheduling and no-shows in service systems
- Two-class routing with admission control and strict priorities
- On the control of a queueing system with aging state information
This page was built for publication: Technical Note—An Equivalence Between Continuous and Discrete Time Markov Decision Processes
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3048616)
