Continuous-time Markov decision processes. Theory and applications
From MaRDI portal
Publication:1041680
zbMath1209.90002MaRDI QIDQ1041680
Xianping Guo, Onésimo Hernández-Lerma
Publication date: 3 December 2009
Published in: Stochastic Modelling and Applied Probability (Search for Journal in Brave)
policycontinuous-time Markov decision processoptimality equationaverage optimalitydiscount optimalitypathwise optimality
Markov and semi-Markov decision processes (90C40) Research exposition (monographs, survey articles) pertaining to operations research and mathematical programming (90-02)
Related Items (only showing first 100 items - show all)
Near-optimal controls of differential systems with switching and random jumps subject to fast switching and wideband noise perturbation ⋮ Optimal control of a multiclass queueing system when customers can change types ⋮ Nonzero-sum constrained discrete-time Markov games: the case of unbounded costs ⋮ Lumpability for uncertain continuous-time Markov chains ⋮ Finite horizon continuous-time Markov decision processes with mean and variance criteria ⋮ Convergence of controlled models and finite-state approximation for discounted continuous-time Markov decision processes with constraints ⋮ Optimization problems in chemical reactions using continuous-time Markov chains ⋮ Resource allocation and routing in parallel multi-server queues with abandonments for cloud profit maximization ⋮ A Jackson network model and threshold policy for joint optimization of energy and delay in multi-hop wireless networks ⋮ Optimal intervention in semi-Markov-based asynchronous probabilistic Boolean networks ⋮ Impulsive control for continuous-time Markov decision processes: a linear programming approach ⋮ Randomized and Relaxed Strategies in Continuous-Time Markov Decision Processes ⋮ State-policy dynamics in evolutionary games ⋮ Discounted stochastic games for continuous-time jump processes with an uncountable state space ⋮ Risk-sensitive discounted cost criterion for continuous-time Markov decision processes on a general state space ⋮ Continuous-time zero-sum games for Markov decision processes with discounted risk-sensitive cost criterion ⋮ Kolmogorov forward equation and explosiveness in countable state Markov processes ⋮ Uniform ergodicity of continuous-time controlled Markov chains: a survey and new results ⋮ Optimal control of Markov processes with age-dependent transition rates ⋮ A continuous-time Markov Stackelberg security game approach for reasoning about real patrol strategies ⋮ Sporadic overtaking optimality in Markov decision problems ⋮ Constrained continuous-time Markov decision processes on the finite horizon ⋮ Constrained Markov decision processes with first passage criteria ⋮ Continuous-time zero-sum stochastic game with stopping and control ⋮ Risk-sensitive continuous-time Markov decision processes with unbounded rates and Borel spaces ⋮ Average cost criterion induced by the regular utility function for continuous-time Markov decision processes ⋮ Imprecise continuous-time Markov chains ⋮ A mean-variance optimization problem for discounted Markov decision processes ⋮ On the optimality of a maintenance queueing system ⋮ Bias optimality of admission control in a non-stationary repairable queue ⋮ Mean-variance problems for finite horizon semi-Markov decision processes ⋮ A minimization problem of the risk probability in first passage semi-Markov decision processes with loss rates ⋮ Nonzero-sum risk-sensitive continuous-time stochastic games with ergodic costs ⋮ Approximation of zero-sum continuous-time Markov games under the discounted payoff criterion ⋮ Zero-sum games for continuous-time Markov jump processes with risk-sensitive finite-horizon cost criterion ⋮ Continuous-time constrained stochastic games with average criteria ⋮ First Passage Optimality and Variance Minimisation of Markov Decision Processes with Varying Discount Factors ⋮ Asymptotic optimality and rates of convergence of quantized stationary policies in continuous-time Markov decision processes ⋮ A probability criterion for zero-sum stochastic games ⋮ The risk probability criterion for discounted continuous-time Markov decision processes ⋮ Nonzero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates ⋮ Reachability and safety objectives in Markov decision processes on long but finite horizons ⋮ Minimizing Ruin Probabilities by Reinsurance and Investment: A Markovian Decision Approach ⋮ The transformation method for continuous-time Markov decision processes ⋮ Continuous-time Markov decision processes under the risk-sensitive average cost criterion ⋮ Finite horizon semi-Markov decision processes with application to maintenance systems ⋮ Discrete-time zero-sum games for Markov chains with risk-sensitive average cost criterion ⋮ First passage models for denumerable semi-Markov decision processes with nonnegative discounted costs ⋮ Realizable Strategies in Continuous-Time Markov Decision Processes ⋮ Analysis of the finite-state ergodic master equation ⋮ Backward stochastic differential equations associated to jump Markov processes and applications ⋮ Continuous-time Markov decision processes under the risk-sensitive first passage discounted cost criterion ⋮ New average optimality conditions for semi-Markov decision processes in Borel spaces ⋮ Optimal maintenance of systems with Markovian mission and deterioration ⋮ Verifiable conditions for average optimality of continuous-time Markov decision processes ⋮ Risk-sensitive control of pure jump process on countable space with near monotone cost ⋮ A risk minimization problem for finite horizon semi-Markov decision processes with loss rates ⋮ Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach ⋮ Stationary equilibria of mean field games with finite state and action space ⋮ Discounted continuous-time constrained Markov decision processes in Polish spaces ⋮ Semi-additive functionals of semi-Markov processes and measure-valued Poisson equation ⋮ Nonzero-sum risk-sensitive finite-horizon continuous-time stochastic games ⋮ On structural properties of the value function for an unbounded jump Markov process with an application to a processor sharing retrial queue ⋮ On the link between infinite horizon control and quasi-stationary distributions ⋮ Risk-sensitive average continuous-time Markov decision processes with unbounded rates ⋮ Parameterized Markov decision process and its application to service rate control ⋮ Markov decision processes with sequential sensor measurements ⋮ An Overview for Markov Decision Processes in Queues and Networks ⋮ Discounted Continuous-Time Controlled Markov Chains: Convergence of Control Models ⋮ Variance minimization of parameterized Markov decision processes ⋮ Continuous-time Markov decision processes with risk-sensitive finite-horizon cost criterion ⋮ Stochastic games for continuous-time jump processes under finite-horizon payoff criterion ⋮ Optimal control of branching diffusion processes: a finite horizon problem ⋮ Customizing exponential semi-Markov decision processes under the discounted cost criterion ⋮ Finite approximation for finite-horizon continuous-time Markov decision processes ⋮ Optimal control of semi-Markov processes with a backward stochastic differential equations approach ⋮ The limit behaviour of imprecise continuous-time Markov chains ⋮ Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes ⋮ Continuous-time Markov decision processes with state-dependent discount factors ⋮ On the vanishing discount factor approach for Markov decision processes with weakly continuous transition probabilities ⋮ Continuous-time stochastic games ⋮ The vanishing discount approach to constrained continuous-time controlled Markov chains ⋮ Towards the optimal control of Markov chains with constraints ⋮ On solutions of Kolmogorov's equations for nonhomogeneous jump Markov processes ⋮ Nonzero-sum games for continuous-time jump processes under the expected average payoff criterion ⋮ Exit time risk-sensitive control for systems of cooperative agents ⋮ Finite-horizon optimality for continuous-time Markov decision processes with unbounded transition rates ⋮ Note on discounted continuous-time Markov decision processes with a lower bounding function ⋮ Approximation of two-person zero-sum continuous-time Markov games with average payoff criterion ⋮ Coupling based estimation approaches for the average reward performance potential in Markov chains ⋮ Constrained expected average stochastic games for continuous-time jump processes ⋮ Optimality of admission control in a repairable queue ⋮ Convergence of Markov decision processes with constraints and state-action dependent discount factors ⋮ Continuous-time zero-sum games with probability criterion ⋮ Mean-semivariance optimality for continuous-time Markov decision processes ⋮ Kolmogorov's equations for jump Markov processes with unbounded jump rates ⋮ Solving the cost to go with time penalization using the Lagrange optimization approach ⋮ Semi-Markov decision processes with variance minimization criterion ⋮ First passage Markov decision processes with constraints and varying discount factors ⋮ A unified approach to time-aggregated Markov decision processes
This page was built for publication: Continuous-time Markov decision processes. Theory and applications
