Discounted continuous-time constrained Markov decision processes in Polish spaces
From MaRDI portal
(Redirected from Publication:655591)
Abstract: This paper is devoted to studying constrained continuous-time Markov decision processes (MDPs) in the class of randomized policies depending on state histories. The transition rates may be unbounded, the reward and costs are admitted to be unbounded from above and from below, and the state and action spaces are Polish spaces. The optimality criterion to be maximized is the expected discounted rewards, and the constraints can be imposed on the expected discounted costs. First, we give conditions for the nonexplosion of underlying processes and the finiteness of the expected discounted rewards/costs. Second, using a technique of occupation measures, we prove that the constrained optimality of continuous-time MDPs can be transformed to an equivalent (optimality) problem over a class of probability measures. Based on the equivalent problem and a so-called -weak convergence of probability measures developed in this paper, we show the existence of a constrained optimal policy. Third, by providing a linear programming formulation of the equivalent problem, we show the solvability of constrained optimal policies. Finally, we use two computable examples to illustrate our main results.
Recommendations
- Continuous-Time Markov Decision Processes with Discounted Rewards: The Case of Polish Spaces
- Constrained optimality for finite horizon semi-Markov decision processes in Polish spaces
- Constrained discounted Markov decision processes with Borel state spaces
- Constrained Continuous-Time Markov Control Processes with Discounted Criteria
- Average optimality for continuous-time Markov decision processes in Polish spaces
- Constrained continuous-time Markov decision processes on the finite horizon
- Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates
- Continuous time Markov decision processes with discounted moment criterion
- Constrained optimality problem of Markov decision processes with Borel spaces and varying discount factors
- Constrained semi-Markov decision processes with ratio and time expected average criteria in Polish spaces
Cites work
- scientific article; zbMATH DE number 1233798 (Why is no real title available?)
- scientific article; zbMATH DE number 1325008 (Why is no real title available?)
- scientific article; zbMATH DE number 1348599 (Why is no real title available?)
- scientific article; zbMATH DE number 700091 (Why is no real title available?)
- scientific article; zbMATH DE number 977051 (Why is no real title available?)
- scientific article; zbMATH DE number 1119444 (Why is no real title available?)
- scientific article; zbMATH DE number 2133327 (Why is no real title available?)
- scientific article; zbMATH DE number 837313 (Why is no real title available?)
- scientific article; zbMATH DE number 1416652 (Why is no real title available?)
- A probabilistic analysis of bias optimality in unichain Markov decision processes
- A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder)
- Average optimality for continuous-time Markov decision processes in Polish spaces
- Bias Optimality in Controlled Queueing Systems
- Computable exponential convergence rates for stochastically ordered Markov processes
- Constrained Average Cost Markov Control Processes in Borel Spaces
- Constrained Continuous-Time Markov Control Processes with Discounted Criteria
- Constrained Discounted Dynamic Programming
- Constrained Discounted Markov Decision Chains
- Constrained Markov Decision Models with Weighted Discounted Rewards
- Constrained Markov control processes in Borel spaces: the discounted case
- Constrained Optimization for Average Cost Continuous-Time Markov Decision Processes
- Constrained admission control to a queueing system
- Constrained continuous-time Markov decision processes with average criteria
- Constrained discounted Markov decision processes and Hamiltonian cycles.
- Constrained dynamic programming with two discount factors: applications and an algorithm
- Constrained markov decision processes with compact state and action spaces: the average case
- Continuous-Time Markov Decision Processes with Discounted Rewards: The Case of Polish Spaces
- Continuous-time Markov decision processes. Theory and applications
- Continuous-time controlled Markov chains.
- Continuously Discounted Markov Decision Model with Countable State and Action Space
- Controlled Markov Models with Countable State Space and Continuous Time
- Discounted Markov decision processes with utility constraints
- Ergodic Control of Continuous-Time Markov Chains with Pathwise Constraints
- Lectures on Choquet's theorem
- Markov Decision Problems and State-Action Frequencies
- Multivariate point processes: predictable projection, Radon-Nikodym derivatives, representation of martingales
- Non-randomized policies for constrained Markov decision processes
- Robustness of Policies in Constrained Markov Decision Processes
- Semi-Markov and Jump Markov Controlled Models: Average Cost Criterion
- Stochastic optimal control. The discrete time case
- Управляемая скачкообразная модель с дисконтированием при наличии ограничений
Cited in
(42)- Zero-sum games for piecewise deterministic Markov decision processes with risk-sensitive finite-horizon cost criterion
- Absorbing continuous-time Markov decision processes with total cost criteria
- Kolmogorov's equations for jump Markov processes and their applications to control problems
- Linear programming and constrained average optimality for general continuous-time Markov decision processes in history-dependent policies
- Finite-horizon piecewise deterministic Markov decision processes with unbounded transition rates
- Risk-sensitive continuous-time stochastic games with the average criterion and a compact state space
- Risk-sensitive average continuous-time Markov decision processes with unbounded transition and cost rates
- Constrained expected average stochastic games for continuous-time jump processes
- Continuous-time Markov decision processes with state-dependent discount factors
- A mean-variance optimization problem for discounted Markov decision processes
- Constrained Continuous-Time Markov Control Processes with Discounted Criteria
- Sufficiency of Markov policies for continuous-time jump Markov decision processes
- Risk-sensitive zero-sum games for continuous-time jump processes with unbounded rates and Borel spaces
- Zero-sum infinite-horizon discounted piecewise deterministic Markov games
- Constrained semi-Markov decision processes with ratio and time expected average criteria in Polish spaces
- Continuous-time zero-sum games for Markov chains with risk-sensitive finite-horizon cost criterion
- On the first passage \(g\)-mean-variance optimality for discounted continuous-time Markov decision processes
- Nonzero-sum risk-sensitive continuous-time stochastic games with ergodic costs
- Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces
- Constrained Markov decision processes with non-constant discount factor
- Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach
- Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach
- Conditions for the solvability of the linear programming formulation for constrained discounted Markov decision processes
- Risk-sensitive continuous-time Markov decision processes with unbounded rates and Borel spaces
- A linear programming formulation for constrained discounted continuous control for piecewise deterministic Markov processes
- Continuous-time zero-sum games for markov decision processes with discounted risk-sensitive cost criterion on a general state space
- First passage Markov decision processes with constraints and varying discount factors
- Continuous-Time Markov Decision Processes with Discounted Rewards: The Case of Polish Spaces
- First passage risk probability minimization for piecewise deterministic Markov decision processes
- Constrained absorbing continuous-time stochastic games
- Continuous-time zero-sum games for Markov decision processes with discounted risk-sensitive cost criterion
- Continuous-time Markov decision processes under the risk-sensitive first passage discounted cost criterion
- Discounted continuous-time Markov decision processes with unbounded rates: the convex analytic approach
- Risk-sensitive discounted continuous-time Markov decision processes with unbounded rates
- Discounted stochastic games for continuous-time jump processes with an uncountable state space
- Convex analytic approach to constrained discounted Markov decision processes with non-constant discount factors
- Asymptotic optimality of quantized stationary policies in continuous-time Markov decision processes with Polish spaces
- A constrained optimization problem with applications to constrained MDPs
- Constrained continuous-time Markov decision processes on the finite horizon
- A dynamic analytic method for risk-aware controlled martingale problems
- Asymptotic optimality and rates of convergence of quantized stationary policies in continuous-time Markov decision processes
- Constrained total undiscounted continuous-time Markov decision processes
This page was built for publication: Discounted continuous-time constrained Markov decision processes in Polish spaces
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q655591)
