A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder)
From MaRDI portal
Publication:997928
DOI10.1007/BF02837562zbMath1278.90427MaRDI QIDQ997928
Xianping Guo, Onésimo Hernández-Lerma, Tomás Prieto-Rumeau
Publication date: 8 August 2007
Published in: Top (Search for Journal in Brave)
bias optimalityaverage rewardcontinuous-time Markov decision processestransition ratescontrolled Markov chainsdiscounted rewardsensitive discount criteriaunbounded reward
Optimal stochastic control (93E20) Markov and semi-Markov decision processes (90C40) Continuous-time Markov processes on discrete state spaces (60J27)
Related Items (33)
Optimal Control of Piecewise Deterministic Markov Processes ⋮ Impulsive control for continuous-time Markov decision processes: a linear programming approach ⋮ Uniform ergodicity of continuous-time controlled Markov chains: a survey and new results ⋮ Optimal control of Markov processes with age-dependent transition rates ⋮ Continuous-time zero-sum stochastic game with stopping and control ⋮ Hamilton-Jacobi-Bellman inequality for the average control of piecewise deterministic Markov processes ⋮ A mean-variance optimization problem for discounted Markov decision processes ⋮ Controlled diffusion processes with Markovian switchings for modeling dynamical engineering systems ⋮ Strong \(n\)-discount and finite-horizon optimality for continuous-time Markov decision processes ⋮ The transformation method for continuous-time Markov decision processes ⋮ Dynamic control of a single-server system with abandonments ⋮ OPTIMAL SWITCHING ON AND OFF THE ENTIRE SERVICE CAPACITY OF A PARALLEL QUEUE ⋮ COUNTABLE STATE MARKOV PROCESSES: NON-EXPLOSIVENESS AND MOMENT FUNCTION ⋮ Average sample-path optimality for continuous-time Markov decision processes in Polish spaces ⋮ Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces ⋮ Realizable Strategies in Continuous-Time Markov Decision Processes ⋮ Total reward criteria for unconstrained/constrained continuous-time Markov decision processes ⋮ Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach ⋮ Discounted continuous-time constrained Markov decision processes in Polish spaces ⋮ Accuracy of fluid approximations to controlled birth-and-death processes: absorbing case ⋮ Continuous-time Markov decision processes with \(n\)th-bias optimality criteria ⋮ On the vanishing discount factor approach for Markov decision processes with weakly continuous transition probabilities ⋮ The vanishing discount approach to constrained continuous-time controlled Markov chains ⋮ Finite-horizon optimality for continuous-time Markov decision processes with unbounded transition rates ⋮ Countable state Markov decision processes with unbounded jump rates and discounted cost: optimality equation and approximations ⋮ Delayed Nondeterminism in Continuous-Time Markov Decision Processes ⋮ New discount and average optimality conditions for continuous-time Markov decision processes ⋮ Risk-sensitive control of continuous time Markov chains ⋮ Bias optimality for multichain continuous-time Markov decision processes ⋮ Variance minimization and the overtaking optimality approach to continuous-time controlled Markov chains ⋮ Impulsive Control for Continuous-Time Markov Decision Processes ⋮ Average optimality for continuous-time Markov decision processes under weak continuity conditions ⋮ Efficient approximation of optimal control for continuous-time Markov games
Cites Work
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Multicriteria impulsive control of jump Markov processes
- Optimal design and control of queues
- Zero-sum continuous-time Markov games with unbounded transition and discounted payoff rates
- Average optimality for continuous-time Markov decision processes in Polish spaces
- Variance minimization and the overtaking optimality approach to continuous-time controlled Markov chains
- Decision and control in uncertain resource systems
- Conditions for existence of average and Blackwell optimal stationary policies in denumerable Markov decision processes
- Continuous-time Markov chains. An applications-oriented approach
- Discounted and average Markov decision processes with unbounded rewards: New conditions
- Continuous time Markovian decision processes average return criterion
- Continuous time control of Markov processes on an arbitrary state space: Discounted rewards
- Mathematical models for the control of pests and infectious diseases: A survey
- Optimal control of the simple stochastic epidemic with variable recovery rates
- Blackwell optimality in the class of stationary policies in Markov decision chains with a Borel state space and unbounded rewards
- Bias optimality versus strong 0-discount optimality in Markov control processes with unbounded costs
- Continuous-time controlled Markov chains with discounted rewards
- The scalarization approach to multiobjective Markov control problems: \textit{Why} does it work?
- Basic ideas for event-based optimization of Markov systems
- The Laurent series, sensitive discount and Blackwell optimality for continuous-time controlled Markov chains
- Continuous time Markov decision processes with discounted moment criterion
- Sensitive discount optimality in controlled one-dimensional diffusions
- Continuous-time controlled Markov chains.
- Computable exponential convergence rates for stochastically ordered Markov processes
- Blackwell optimality in the class of all policies in Markov decision chains with a Borel state space and unbounded rewards
- On maximal rewards and \(\varepsilon\)-optimal policies in continuous time Markov decision chains
- Blackwell optimality in the class of Markov policies for continuous-time controlled Markov chains
- Finite state continuous time Markov decision processes with an infinite planning horizon
- Bias and overtaking equilibria for zero-sum continuous-time Markov games
- Technical Note—An Equivalence Between Continuous and Discrete Time Markov Decision Processes
- Stability of Markovian processes III: Foster–Lyapunov criteria for continuous-time processes
- Bias optimal admission control policies for a multiclass nonstationary queueing system
- Denumerable-state continuous-time Markov decision processes with unbounded transition and reward rates under the discounted criterion
- Bias Optimality for Continuous-Time Controlled Markov Chains
- Semi-Markov and Jump Markov Controlled Models: Average Cost Criterion
- Average, Sensitive and Blackwell Optimal Policies in Denumerable Markov Decision Chains with Unbounded Rewards
- On Homogeneous Markov Models with Continuous Time and Finite or Countable State Space
- Optimal Control of a Birth and Death Epidemic Process
- Recurrence Conditions for Average and Blackwell Optimality in Denumerable State Markov Decision Chains
- On the Second Optimality Equation for Semi-Markov Decision Models
- Applying a New Device in the Optimization of Exponential Queuing Systems
- On a Class of Strategies in General Markov Decision Models
- Relation between continuous and discrete time markovian decision problems
- Controlled Markov Models with Countable State Space and Continuous Time
- A birth–death model of advertising and pricing
- Blackwell Optimality in Borelian Continuous-in-Action Markov Decision Processes
- Bias Optimality in Controlled Queueing Systems
- Zero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates
- A note on bias optimality in controlled queueing systems
- A probabilistic analysis of bias optimality in unichain Markov decision processes
- A note on optimality conditions for continuous-time Markov decision processes with average cost criterion
- Blackwell optimal policies in a Markov decision process with a Borel state space
- Overtaking and Almost-Sure Optimality for Infinite Horizon Markov Decision Processes
- BIAS OPTIMALITY IN A QUEUE WITH ADMISSION CONTROL
- Nondiscounted Continuous Time Markovian Decision Process with Countable State Space
- Drift and monotonicity conditions for continuous-time controlled markov chains with an average criterion
- Optimal Control of Ergodic Continuous-Time Markov Chains with Average Sample-Path Rewards
- Ergodic Control of Continuous-Time Markov Chains with Pathwise Constraints
- Discrete Dynamic Programming
- On Finding Optimal Policies in Discrete Dynamic Programming with No Discounting
- On Recurrent Denumerable Decision Processes
- Discrete Dynamic Programming with a Small Interest Rate
- Discrete Dynamic Programming with Sensitive Discount Optimality Criteria
- Continuously Discounted Markov Decision Model with Countable State and Action Space
- Markov Decision Processes with Finite State and Decision Spaces
- Nonzero-sum games for continuous-time Markov chains with unbounded discounted payoffs
- On the Equivalence of Two Expected Average Cost Criteria for Semi-Markov Control Processes
- On the Integro-Differential Equations of Purely Discontinuous Markoff Processes
- Nonstationary continuous-time Markov control processes with discounted costs on infinite horizon
- Handbook of Markov decision processes. Methods and applications
This page was built for publication: A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder)
