A survey of Markov decision models for control of networks of queues

DOI10.1007/BF01158935zbMath0772.90082OpenAlexW2055959255MaRDI QIDQ1801813

Shaler jun. Stidham, Richard R. Weber

Publication date: 17 August 1993

Published in: Queueing Systems (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1007/bf01158935

zbMATH Keywords

optimal control of networks of queuesstructure of optimal control policies

Mathematics Subject Classification ID

Markov and semi-Markov decision processes (90C40) Research exposition (monographs, survey articles) pertaining to operations research and mathematical programming (90-02) Applications of Markov renewal processes (reliability, queueing networks, etc.) (60K20)

Related Items (49)

Asymptotic optimality of tracking policies in stochastic networks. ⋮ Balancing herding and congestion in service systems: a queueing perspective ⋮ Optimal control of a multiclass queueing system when customers can change types ⋮ Controlled semi-Markov chains with risk-sensitive average cost criterion ⋮ FINDING EXPECTED REVENUES IN G-NETWORK WITH MULTIPLE CLASSES OF POSITIVE AND NEGATIVE CUSTOMERS ⋮ OPTIMAL PRODUCTION POLICIES WITH MULTISTAGE STOCHASTIC DEMAND LEAD TIMES ⋮ Optimal threshold policies in a two-class preemptive priority queue with admission and termination control ⋮ Scheduling in a multi-class series of queues with deterministic service times ⋮ A perturbation approach to a class of discounted approximate value iteration algorithms with Borel spaces ⋮ The shorter queue polling model ⋮ On computing average cost optimal policies with application to routing to parallel queues ⋮ Optimal threshold control of a retrial queueing system with finite buffer ⋮ Forecasting anticipated incomes in the Markov networks with positive and negative customers ⋮ Optimal control of a queue under a quality-of-service constraint with bounded and unbounded rates ⋮ User equilibria for a parallel queueing system with state dependent routing ⋮ Markov Reward Models and Markov Decision Processes in Discrete and Continuous Time: Performance Evaluation and Optimization ⋮ Optimality of monotonic policies for two-action Markovian decision processes, with applications to control of queues with delayed information ⋮ A new method of proving structural properties for certain class of stochastic dynamic control problems ⋮ Optimal switching strategy between admission control and pricing control policies with two types of customers and search costs ⋮ CUSTOMER SELECTION PROBLEM WITH SEARCH COST, DUE DATE, SIDELINE PROFIT, AND NO WAITING ROOM ⋮ Average criteria in denumerable semi-Markov decision chains under risk-aversion ⋮ OPERATOR ANALYSIS OF KOLMOGOROV–CHAPMAN TYPE EQUATIONS WITH DIFFERENTIAL OPERATOR ⋮ Cost-aware defense for parallel server systems against reliability and security failures ⋮ Construction and regularity of transition functions on Polish spaces under measurability conditions ⋮ Monotonicity properties of user equilibrium policies for parallel batch systems ⋮ Optimal admission control in a queueing system with heterogeneous traffic. ⋮ Monotone optimal control for a class of Markov decision processes ⋮ Maximizing throughput in finite-source parallel queue systems ⋮ Performance optimization of queueing systems with perturbation realization ⋮ TWO-CLASS ROUTING WITH ADMISSION CONTROL AND STRICT PRIORITIES ⋮ Examining military medical evacuation dispatching policies utilizing a Markov decision process model of a controlled queueing system ⋮ Existence and regularity of a nonhomogeneous transition matrix under measurability conditions ⋮ An Overview for Markov Decision Processes in Queues and Networks ⋮ A Poisson equation for the risk-sensitive average cost in semi-Markov chains ⋮ ROUTING OF AIRPLANES TO TWO RUNWAYS: MONOTONICITY OF OPTIMAL CONTROLS ⋮ Controlling load distribution in the queuing systems ⋮ Calculating expected incomes in open Markov networks with requests of different classes and different peculiarities ⋮ Denumerable controlled Markov chains with average reward criterion: Sample path optimality ⋮ Optimal admission and pricing control problem with deterministic service times and sideline profit ⋮ On the dynamic allocation of assets subject to failure ⋮ Equilibrium points and equilibrium sets of some \(GI /M/1\) queues ⋮ On the structure of value functions for threshold policies in queueing models ⋮ Admission control strategies for tandem Markovian loss systems ⋮ Contractive approximations in risk-sensitive average semi-Markov decision chains on a finite state space ⋮ On basic operations related to network induction of discrete convex functions ⋮ FINDING EXPECTED REVENUES IN G-NETWORK WITH SIGNALS AND CUSTOMERS BATCH REMOVAL ⋮ Dynamic safety-stocks for asymptotic optimality in stochastic networks ⋮ Optimal reservation policy for two queues in tandem. ⋮ Near optimal admission control for multiserver loss queues in series

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• K competing queues with geometric service requirements and linear costs: The \(\mu\) c-rule is always optimal
• Control of arrivals to two queues in series
• A note on Optimal control of a queueing system with two heterogeneous servers
• Martingale dynamics and optimal routing in a network
• Necessary conditions for the optimality equation in average-reward Markov decision processes
• Interchange arguments for classical scheduling problems in queues
• Optimality of the shortest line discipline with state-dependent service rates
• Recent results on conditions for the existence of average optimal stationary policies
• Optimality of routing and servicing in dependent parallel processing systems
• Interchange arguments in stochastic scheduling
• Optimal use of an extra server in a two station tandem queueing network
• Two competing queues with linear costs and geometric service requirements: the μc-rule is often optimal
• Optimal control of two interacting service stations
• Optimal control of a queueing system with two heterogeneous servers
• On the optimality of the shortest line discipline
• Average Cost Semi-Markov Decision Processes and the Control of Queueing Systems
• Dynamic Scheduling of a Four-Station Queueing Network
• On the Shortest Queue Policy for the Tandem Parallel Queue
• Optimal control of admission to a quenching system
• Extensions of the multiarmed bandit problem: The discounted case
• Extremal Splittings of Point Processes
• Deciding Which Queue to Join: Some Counterexamples
• Comparison of Policies for Routing Customers to Parallel Queueing Systems
• Control of Markov Chains with Long-Run Average Cost Criterion: The Dynamic Programming Equations
• Routing in queueing networks under imperfect information: stochastic dominance and thresholds
• Average Cost Optimal Stationary Policies in Infinite State Markov Decision Processes with Unbounded Costs
• A simple dynamic routing problem
• On decentralized dynamic routing for congested traffic networks
• Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal
• A Priority Queue with Discounted Linear Costs
• Dynamic Scheduling of a Multiclass Queue: Discount Optimality
• Optimality of the shortest line discipline
• Optimal Control of Single-Server Queuing Networks and Multi-Class M/G/1 Queues with Feedback
• On the optimal assignment of customers to parallel servers
• Time-Sharing Service Systems. I
• Minimizing a Submodular Function on a Lattice
• A Basic Dynamic Routing Problem and Diffusion
• Monotone Optimal Control of Permutable GSMPs
• Optimal control of service rates in networks of queues
• Optimal Control of a Service Facility with Variable Exponential Service Times and Constant Arrival Rate

This page was built for publication: A survey of Markov decision models for control of networks of queues