An Overview for Markov Decision Processes in Queues and Networks

DOI10.1007/978-981-15-0864-6_3zbMath1473.90175arXiv1907.10243MaRDI QIDQ3305576

Jing-Yu Ma, Rui-Na Fan, Li Xia, Quan-Lin Li

Publication date: 10 August 2020

Published in: Stochastic Models in Reliability, Network Security and System Safety (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1907.10243

zbMATH Keywords

Markov decision processes; queueing networks; queueing systems; sensitivity-based optimization; event-based optimization

Mathematics Subject Classification ID

90B22: Queues and service in operations research

90B10: Deterministic network models in operations research

90C40: Markov and semi-Markov decision processes

90-02: Research exposition (monographs, survey articles) pertaining to operations research and mathematical programming

Related Items

A complete algebraic solution to the optimal dynamic rationing policy in the stock-rationing queue with two demand classes

Uses Software

• UMDES

Cites Work

• Optimal pricing for a \(\mathrm{GI}/\mathrm{M}/k/N\) queue with several customer types and holding costs
• Optimal control of a multiclass queueing system when customers can change types
• Nonlinear Markov processes in big networks
• Optimal policies of \(M(t)/M/c/c\) queues with two different levels of servers
• On a doubly dynamically controlled supermarket model with impatient customers
• Dynamic control of a flexible server in an assembly-type queue with setup costs
• Maximizing throughput in finite-source parallel queue systems
• Performance optimization of queueing systems with perturbation realization
• A tutorial on event-based optimization -- a new optimization framework
• Event-based optimization of admission control in open queueing networks
• Control and observation for dynamical queueing networks. I
• Optimal arrival rate and service rate control of multi-server queues
• On structural properties of the value function for an unbounded jump Markov process with an application to a processor sharing retrial queue
• The mean-field computation in a supermarket model with server multiple vacations
• Parameterized Markov decision process and its application to service rate control
• Block-structured supermarket models
• Markov decision processes in practice
• Optimal control of queueing systems with heterogeneous servers
• Dynamic admission and service rate control of a queue
• Queueing networks. A fundamental approach
• Admission control and pricing in a queue with batch arrivals
• Dynamic control of a single-server system with abandonments
• A mean field approach for optimization in discrete time
• Markov chains and stochastic stability
• Bicriteria optimization of a queue with a controlled input stream
• On the optimality of a full-service policy for a queueing system with discounted costs
• Optimal adaptive control policy for joint machine maintenance and product quality control
• Optimal threshold policies in a two-class preemptive priority queue with admission and termination control
• Stability of queueing networks. École d'Été de probabilités de Saint-Flour XXXVI-2006
• Optimal balking strategies and pricing for the single server Markovian queue with compartmented waiting space
• Continuous-time Markov decision processes. Theory and applications
• Control of arrivals to two queues in series
• A Markov decision algorithm for optimal inspections and revisions in a maintenance system with partial information
• Optimal dynamic routing in Markov queueing networks
• Optimal control for an \(M^X/G/1\) queue with two services modes
• Controlling the GI/M/1 queue by conditional acceptance of customers
• Loss networks
• Queueing network models for load balancing in distributed systems
• Optimality of routing and servicing in dependent parallel processing systems
• Optimal dynamic load distribution in a class of flow-type flexible manufacturing systems
• Continuous-time Markov chains. An applications-oriented approach
• Design of manufacturing systems using queueing models
• Monotone control of queueing networks
• Stochastic differential systems, stochastic control theory and applications. (Proceedings of a workshop, held at IMA, Minnesota University, Minneapolis, June 9-19, 1986)
• Continuous time control of the arrival process in an M/G/1 queue
• Pricing, routing, and incentive compatibility in multiserver queues
• Dynamic scheduling in multiclass queueing networks: Stability under discrete-review policies
• Realization probabilities. The dynamics of queuing systems
• Optimization of multiclass queueing networks: Polyhedral and nonlinear characterizations of achievable performance
• Control and scheduling in a two-station queueing network: Optimal policies and heuristics
• Scheduling in a multi-class series of queues with deterministic service times
• Non zero-sum stochastic games in admission, service and routing control in queueing systems
• Optimal service control against worst case admission policies: A multichained stochastic game
• On the optimal control of arrivals to a single queue with arbitrary feedback delay
• A two-stage tandem queue attended by a moving server with holding and switching costs
• On the bias vector of a two-class preemptive priority queue
• Discrete-event control of stochastic networks: multimodularity and regularity.
• Large-deviations analysis of the fluid approximation for a controllable tandem queue
• On the value function of a priority queue with an application to a controlled polling model
• Dynamic control of a tandem system with abandonments
• An algorithmic approach for sensitivity analysis of perturbed quasi-birth-and-death processes
• Basic ideas for event-based optimization of Markov systems
• A survey of Markov decision models for control of networks of queues
• Value iteration and optimization of multiclass queueing networks
• Optimal intensity control of a multi-class queue
• Queueing models of call centers: An introduction
• Optimality of \(D\)-policies for an \(M\)/\(G\)/1 queue with a removable server
• Asymptotic optimality of tracking policies in stochastic networks.
• Optimal pricing and admission control in a queueing system with periodically varying param\-eters
• Scheduling a multi class queue with many exponential servers: asymptotic optimality in heavy traffic.
• On positive Harris recurrence of multiclass queueing networks: A unified approach via fluid limit models
• Optimal control of multiclass parallel service systems
• Optimal control of polling models for transportation applications
• Optimality of monotonic policies for two-action Markovian decision processes, with applications to control of queues with delayed information
• Sample path methods in the control of queues
• Zero-sum Markov games and worst-case optimal control of queueing systems
• Dynamic scheduling of a single-server two-class queue with constant retrial policy
• Structural properties of the optimal resource allocation policy for single-queue systems
• Optimal energy-efficient policies for data centers through sensitivity-based optimization
• Optimal control of a queueing network system with two types of customers
• Markov decision processes with their applications
• Service rate control of closed Jackson networks from game theoretic perspective
• Priority option pricing in an \(M/M/m\) queue
• Dynamic load balancing in parallel queueing systems: stability and optimal control
• Optimal admission policies for a finite queue with bursty arrivals
• Simultaneous optimization of flow control and scheduling in a single server queue with two job classes
• Simultaneous optimization of flow-control and scheduling in a single server queue with two job classes: Numerical results and approximation
• The Complexity of Optimal Queuing Network Control
• M/M/1 Queues with Switching Costs and Hysteretic Optimal Control
• The Underlying Markov Decision Process in the Single-Leg Airline Yield-Management Problem
• Partially Observed Markov Decision Processes
• Planning with Markov Decision Processes: An AI Perspective
• Admission Control to an M/M/1 Queue with Partial Information
• OPTIMALITY OF TRUNK RESERVATION FOR AN M/M/K/N QUEUE WITH SEVERAL CUSTOMER TYPES AND HOLDING COSTS
• Technical Note—Optimal Structural Results for Assemble-to-Order Generalized M-Systems
• Optimal Control of an Inventory System With Joint Production and Pricing Decisions
• Optimization of Average Rewards of Time Nonhomogeneous Markov Chains
• Perturbation realization, potentials, and sensitivity analysis of Markov processes
• Dynamic Scheduling Rules for a Multiproduct Make-to-Stock Queue
• Bias Optimality in Controlled Queueing Systems
• CONVEXITY IN TANDEM QUEUES
• Optimal control of a queueing system with heterogeneous servers and setup costs
• On optimal call admission control in resource-sharing system
• Dynamic Pricing Control for Open Queueing Networks
• Semismooth Potentials of Stochastic Systems With Degenerate Diffusions
• Optimality Conditions for Long-Run Average Rewards With Underselectivity and Nonsmooth Features
• Optimal Control of State-Dependent Service Rates in a MAP/M/1 Queue
• Load balancing problems for multiclass jobs in distributed/parallel computer systems
• Near-Optimal Switching Strategies for a Tandem Queue
• Dynamic service rate control for a single-server queue with Markov-modulated arrivals
• Applied Probability and Queues
• Heavy Traffic Analysis of a Controlled Multiclass Queueing Network via Weak Convergence Methods
• Optimal control of service rates in networks of queues
• Optimal Pricing for an Unbounded Queue
• OPTIMAL CONTROL OF A TWO-STAGE TANDEM QUEUING SYSTEM WITH FLEXIBLE SERVERS
• Optimal control of arrivals to queues with delayed queue length information
• Advances in Queueing Theory, Methods, and Open Problems
• Throughput properties of a queueing network with distributed dynamic routing and flow control
• On the Pathwise Optimal Bernoulli Routing Policy for Homogeneous Parallel Servers
• BIAS OPTIMALITY IN A QUEUE WITH ADMISSION CONTROL
• Effects of System Parameters on the Optimal Cost and Policy in a Class of Multidimensional Queueing Control Problems
• A Uniformization Approach for the Dynamic Control of Queueing Systems with Abandonments
• Throughput Scalability Analysis of Fork-Join Queueing Networks
• The Supermarket Game
• Dynamic scheduling in single-server multiclass service systems with unit buffers
• On Solving Event-Based Optimization With Average Reward Over Infinite Stages
• Networked Markov Decision Processes With Delays
• Mean Field for Markov Decision Processes: From Discrete to Continuous Optimization
• Optimal Admission Control for Tandem Queues With Loss
• Max-Min Optimality of Service Rate Control in Closed Queueing Networks
• OPTIMAL SWITCHING ON AND OFF THE ENTIRE SERVICE CAPACITY OF A PARALLEL QUEUE
• Networks of Waiting Lines
• Priority and dynamic scheduling in a make‐to‐stock queue with hyperexponential demand
• Monotonicity in Markov Reward and Decision Chains: Theory and Applications
• A New Policy Evaluation Algorithm for Markov Decision Processes with Quasi Birth-Death Structure
• Queueing Networks and Markov Chains
• Bicriterion Optimization of an M/G/1 Queue with A Removable Server
• The optimal admission policy to a multiserver queue with finite horizon
• Optimal service-rate selection in an $M| G |\hat 1$Queue
• Customers' Optimal Joining Rules for the GI/M/s Queue
• Optimal Control of a Service Facility with Variable Exponential Service Times and Constant Arrival Rate
• On the Optimality of Single-Server Queuing Systems
• Optimal control of batch service queues
• Optimal Control of a Single-Server Queue with Balking and Reneging
• Optimal Control of the Vacation Scheme in an M/G/1 Queue
• Analysis of queueing networks with blocking
• Monotone control of queueing systems with heterogeneous servers
• Queueing networks with discrete time scale. Explicit expressions for the steady state behaviour of discrete time stochastic networks
• A multiclass queue in heavy traffic with throughput time constraints: Asymptotically optimal dynamic controls
• Computation of a near-optimal service policy for a single-server queue with homogeneous jobs
• Handbook of Markov decision processes. Methods and applications
• Task allocation in multiserver systems -- a survey of results.
• Admission control for a multi-server queue with abandonment
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Customer routing to different servers with complete information
• Optimal and Approximately Optimal Control Policies for Queues in Heavy Traffic
• Revenue Optimization for a Make-to-Order Queue in an Uncertain Market Environment
• Dynamic Lead-Time Quotation for an M/M/1 Base-Stock Inventory Queue
• Stock Rationing in an M/E_k/1 Make-to-Stock Queue
• Dynamic Pricing and Lead-Time Quotation for a Multiclass Make-to-Order Queue
• A single sample path-based performance sensitivity formula for Markov chains
• Optimal use of an extra server in a two station tandem queueing network
• Optimal control of queueing networks: an approach via fluid models
• Optimal Routing Among ⋅/M/1 Queues with Partial Information
• Dynamic production control in a serial line with process queue time constraint
• An Efficient Algorithm for Computing Optimal (s, S) Policies
• Optimal control of two interacting service stations
• Optimal control of entry to anM/Ek/1 queue serving several classes of customers
• Optimal Entering Rules for a Customer with Wait Option at an M/G/1 Queue
• Some properties of optimal control policies for entry to an M/M/1 queue
• Optimal admission pricing and service rate control of anM[x/M/s queue with reneging]
• Optimal control of arrivals to multiserver queues in a random environment
• Optimal control of a queueing system with two heterogeneous servers
• M/M/1 Queueing Decision Processes with Monotone Hysteretic Optimal Policies
• Optimality of threshold policies in single-server queueing systems with server vacations
• Optimal monotone hysteretic Markov policies in anM/M/1 queueing model with switching costs and finite time horizon
• Dynamic Control of a Multiclass Queue with Thin Arrival Streams
• Average Cost Semi-Markov Decision Processes and the Control of Queueing Systems
• On the Shortest Queue Policy for the Tandem Parallel Queue
• Introduction to Discrete Event Systems
• ADMISSION CONTROL WITH INCOMPLETE INFORMATION TO A FINITE BUFFER QUEUE
• ON THE OPTIMAL OPEN-LOOP CONTROL POLICY FOR DETERMINISTIC AND EXPONENTIAL POLLING SYSTEMS
• Constructive Computation in Stochastic Models with Applications
• Scheduling policies in the M/G/1 make-to-stock queue
• DYNAMIC ROUTING POLICIES FOR MULTISKILL CALL CENTERS
• Optimal Design of Queueing Systems
• Dynamic Control of a Queue with Adjustable Service Rate
• Analysis, Design, and Control of Queueing Systems
• Optimal service-rate control of M/G/1 queueing systems using phase methods
• Control of a Heterogeneous Two-Server Exponential Queueing System
• Average optimal policies in Markov decision drift processes with applications to a queueing and a replacement model
• Computing Optimal Control Limits for GI/M/S Queuing Systems with Controlled Arrivals
• Adaptive control of M/M/1 queues—continuous-time Markov decision process approach
• Optimal control of admission to a quenching system
• Queueing Networks: A Survey of Their Random Processes
• The Bulk Service Queue with a General Control Strategy: Theoretical Analysis and a New Computational Procedure
• SERVICE MECHANISM CONTROL AND ARRIVAL CONTROL OF A TWO-STATION TANDEM QUEUE
• Perturbation theory for Markov reward processes with applications to queueing systems
• Decentralized control of service rates in a closed Jackson network
• Dynamic priority rules for cyclic-type queues
• Constrained admission control to a queueing system
• On monotone optimal policies in a queueing model ofM/G/1 type with controllable service time distribution
• Optimal control of random walks, birth and death processes, and queues
• An M/M/1 Dynamic Priority Queue with Optional Promotion
• Optimal control for entry of many classes of customers to anM/M/1 queue
• Socially and Individually Optimal Control of Arrivals to a GI/M/1 Queue
• Optimal control of service in tandem queues
• Optimal control of an M/G/1 queue with imperfectly observed queue length when the input source is finite
• Optimal control of admission to a multiserver queue with two arrival streams
• Optimal routing and buffer allocation for a class of finite capacity queueing systems
• An Efficient Algorithm for Computing an Optimal (r, Q) Policy in Continuous Review Stochastic Inventory Systems
• Optimal Dynamic Assignment of Customers to Heterogeneous Servers in Parallel
• Optimal Dynamic Pricing Policies for an M/M/s Queue
• Applying a New Device in the Optimization of Exponential Queuing Systems
• Open, Closed, and Mixed Networks of Queues with Different Classes of Customers
• Dynamic Scheduling of a Multiclass Queue: Discount Optimality
• A Note on Optimal Service Selection in a Single Server Queue
• Networks of queues
• Markov decision chains with unbounded costs and applications to the control of queues
• Optimal control of batch service queues with switching costs
• A Classified Bibliography of Research on Optimal Design and Control of Queues
• Optimization of Priority Queues—A Semi-Markov Decision Chain Approach
• On the optimal assignment of customers to parallel servers
• Optimal control of the service rate in an M/G/1 queueing system
• Stochastic monotonicity in general queueing networks
• Introduction to Matrix Analytic Methods in Stochastic Modeling
• On the optimality of LEPT and μc rules for parallel processors and dependent arrival processes
• Optimal control of the M/G/1 queue with repeated vacations of the server
• A direct approach to decentralized control of service rates in a closed Jackson network
• Dynamic routing and jockeying controls in a two-station queueing system
• Inventory Rationing in a Make-to-Stock Production System with Several Demand Classes and Lost Sales
• Stock-rationing policy for a make-to-stock production system with two priority classes and backordering