A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder)

DOI10.1007/BF02837562zbMATH Open1278.90427MaRDI QIDQ997928FDOQ997928

Xianping Guo, Onésimo Hernández-Lerma, Tomás Prieto-Rumeau

Publication date: 8 August 2007

Published in: Top (Search for Journal in Brave)

Recommendations

• Continuous-time Markov decision processes. Theory and applications
• Continuous-time markov decision processes with nonzero terminal reward
• A Survey of Applications of Markov Decision Processes
• A survey on semi-Markov decision processes
• Continuous-Time Markov Decision Processes with Unbounded Transition and Discounted-Reward Rates
• Continuous-time Markov decision processes with \(n\)th-bias optimality criteria
• scientific article
• Continuous time markov decision processes with nonuniformly bounded transition rate: expected total rewards

zbMATH Keywords

bias optimalityaverage rewardcontinuous-time Markov decision processestransition ratescontrolled Markov chainsdiscounted rewardsensitive discount criteriaunbounded reward

Mathematics Subject Classification ID

Continuous-time Markov processes on discrete state spaces (60J27) Markov and semi-Markov decision processes (90C40) Optimal stochastic control (93E20)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Bias Optimality in Controlled Queueing Systems
• Title not available (Why is that?)
• BIAS OPTIMALITY IN A QUEUE WITH ADMISSION CONTROL
• Title not available (Why is that?)
• Bias optimality versus strong 0-discount optimality in Markov control processes with unbounded costs
• Basic ideas for event-based optimization of Markov systems
• The Laurent series, sensitive discount and Blackwell optimality for continuous-time controlled Markov chains
• Sensitive discount optimality in controlled one-dimensional diffusions
• Finite state continuous time Markov decision processes with an infinite planning horizon
• Bias and overtaking equilibria for zero-sum continuous-time Markov games
• Title not available (Why is that?)
• Stability of Markovian processes III: Foster–Lyapunov criteria for continuous-time processes
• Title not available (Why is that?)
• Zero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates
• Nonzero-sum games for continuous-time Markov chains with unbounded discounted payoffs
• Zero-sum continuous-time Markov games with unbounded transition and discounted payoff rates
• Continuous-time Markov chains. An applications-oriented approach
• Title not available (Why is that?)
• Technical Note—An Equivalence Between Continuous and Discrete Time Markov Decision Processes
• Denumerable-state continuous-time Markov decision processes with unbounded transition and reward rates under the discounted criterion
• Dynamic programming and optimal control. Vol. 2.
• On the Second Optimality Equation for Semi-Markov Decision Models
• Title not available (Why is that?)
• Discrete Dynamic Programming
• Title not available (Why is that?)
• Discrete Dynamic Programming with Sensitive Discount Optimality Criteria
• Semi-Markov and Jump Markov Controlled Models: Average Cost Criterion
• Title not available (Why is that?)
• Controlled Markov Models with Countable State Space and Continuous Time
• Ergodic Control of Continuous-Time Markov Chains with Pathwise Constraints
• Multicriteria impulsive control of jump Markov processes
• Applying a New Device in the Optimization of Exponential Queuing Systems
• Optimal design and control of queues
• Computable exponential convergence rates for stochastically ordered Markov processes
• Blackwell optimality in the class of Markov policies for continuous-time controlled Markov chains
• Bias Optimality for Continuous-Time Controlled Markov Chains
• Average, Sensitive and Blackwell Optimal Policies in Denumerable Markov Decision Chains with Unbounded Rewards
• Optimal Control of a Birth and Death Epidemic Process
• Recurrence Conditions for Average and Blackwell Optimality in Denumerable State Markov Decision Chains
• Drift and monotonicity conditions for continuous-time controlled markov chains with an average criterion
• Handbook of Markov decision processes. Methods and applications
• Average optimality for continuous-time Markov decision processes in Polish spaces
• On Finding Optimal Policies in Discrete Dynamic Programming with No Discounting
• Discounted and average Markov decision processes with unbounded rewards: New conditions
• Blackwell optimality in the class of all policies in Markov decision chains with a Borel state space and unbounded rewards
• Title not available (Why is that?)
• Mathematical models for the control of pests and infectious diseases: A survey
• On the Equivalence of Two Expected Average Cost Criteria for Semi-Markov Control Processes
• Title not available (Why is that?)
• Optimal Control of Ergodic Continuous-Time Markov Chains with Average Sample-Path Rewards
• Continuously Discounted Markov Decision Model with Countable State and Action Space
• Continuous-time controlled Markov chains.
• On the Integro-Differential Equations of Purely Discontinuous Markoff Processes
• Nonstationary continuous-time Markov control processes with discounted costs on infinite horizon
• On maximal rewards and \(\varepsilon\)-optimal policies in continuous time Markov decision chains
• Continuous-time controlled Markov chains with discounted rewards
• A note on bias optimality in controlled queueing systems
• Title not available (Why is that?)
• A note on optimality conditions for continuous-time Markov decision processes with average cost criterion
• Nondiscounted Continuous Time Markovian Decision Process with Countable State Space
• Continuous time Markov decision processes with discounted moment criterion
• A probabilistic analysis of bias optimality in unichain Markov decision processes
• Continuous time control of Markov processes on an arbitrary state space: Discounted rewards
• Decision and control in uncertain resource systems
• Variance minimization and the overtaking optimality approach to continuous-time controlled Markov chains
• Discrete Dynamic Programming with a Small Interest Rate
• Overtaking and Almost-Sure Optimality for Infinite Horizon Markov Decision Processes
• On Homogeneous Markov Models with Continuous Time and Finite or Countable State Space
• Title not available (Why is that?)
• Continuous time Markovian decision processes average return criterion
• Title not available (Why is that?)
• Conditions for existence of average and Blackwell optimal stationary policies in denumerable Markov decision processes
• Optimal control of the simple stochastic epidemic with variable recovery rates
• Blackwell optimality in the class of stationary policies in Markov decision chains with a Borel state space and unbounded rewards
• The scalarization approach to multiobjective Markov control problems: \textit{Why} does it work?
• Bias optimal admission control policies for a multiclass nonstationary queueing system
• On a Class of Strategies in General Markov Decision Models
• Relation between continuous and discrete time markovian decision problems
• A birth–death model of advertising and pricing
• Blackwell Optimality in Borelian Continuous-in-Action Markov Decision Processes
• Title not available (Why is that?)
• Blackwell optimal policies in a Markov decision process with a Borel state space
• On Recurrent Denumerable Decision Processes
• Markov Decision Processes with Finite State and Decision Spaces

Cited In (34)

• Optimal Control of Piecewise Deterministic Markov Processes
• Efficient approximation of optimal control for continuous-time Markov games
• Bias optimality for multichain continuous-time Markov decision processes
• Average optimality for continuous-time Markov decision processes under weak continuity conditions
• A mean-variance optimization problem for discounted Markov decision processes
• On the vanishing discount factor approach for Markov decision processes with weakly continuous transition probabilities
• Dynamic control of a single-server system with abandonments
• Variance minimization and the overtaking optimality approach to continuous-time controlled Markov chains
• Continuous-time zero-sum stochastic game with stopping and control
• COUNTABLE STATE MARKOV PROCESSES: NON-EXPLOSIVENESS AND MOMENT FUNCTION
• Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces
• Finite-horizon optimality for continuous-time Markov decision processes with unbounded transition rates
• Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach
• Impulsive Control for Continuous-Time Markov Decision Processes
• Impulsive control for continuous-time Markov decision processes: a linear programming approach
• Continuous-time Markov decision processes with \(n\)th-bias optimality criteria
• Accuracy of fluid approximations to controlled birth-and-death processes: absorbing case
• Discounted continuous-time constrained Markov decision processes in Polish spaces
• Controlled diffusion processes with Markovian switchings for modeling dynamical engineering systems
• The transformation method for continuous-time Markov decision processes
• Uniform ergodicity of continuous-time controlled Markov chains: a survey and new results
• Hamilton-Jacobi-Bellman inequality for the average control of piecewise deterministic Markov processes
• Average sample-path optimality for continuous-time Markov decision processes in Polish spaces
• OPTIMAL SWITCHING ON AND OFF THE ENTIRE SERVICE CAPACITY OF A PARALLEL QUEUE
• Total reward criteria for unconstrained/constrained continuous-time Markov decision processes
• Delayed Nondeterminism in Continuous-Time Markov Decision Processes
• New discount and average optimality conditions for continuous-time Markov decision processes
• Optimal control of Markov processes with age-dependent transition rates
• Title not available (Why is that?)
• Realizable Strategies in Continuous-Time Markov Decision Processes
• The vanishing discount approach to constrained continuous-time controlled Markov chains
• Countable state Markov decision processes with unbounded jump rates and discounted cost: optimality equation and approximations
• Risk-sensitive control of continuous time Markov chains
• Strong \(n\)-discount and finite-horizon optimality for continuous-time Markov decision processes

This page was built for publication: A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder)

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q997928)