Markov Decision Problems and State-Action Frequencies

DOI10.1137/0329043zbMath0733.90075MaRDI QIDQ3360681

Adam Shwartz, Eitan Altman

Publication date: 1991

Published in: SIAM Journal on Control and Optimization (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1137/0329043

zbMATH Keywords

adaptive control; constrained optimization; countable state and action spaces; controlled Markov chain; state-action frequencies; stationary policies; average cost functionals; nonstationary control; variability-sensitive optimization

Mathematics Subject Classification ID

90B50: Management decision making, including multiple objectives

60K25: Queueing theory (aspects of probability theory)

90B22: Queues and service in operations research

60J10: Markov chains (discrete-time Markov processes on discrete state spaces)

93E20: Optimal stochastic control

60G17: Sample path properties

90C40: Markov and semi-Markov decision processes

Related Items

Constrained Semi-Markov decision processes with average rewards, Linear programming formulation of MDPs in countable state space: The multichain case, The Linear Program approach in multi-chain Markov Decision Processes revisited, Discrete-Time Semi-Markov Random Evolutions and their Applications, Asymptotic properties of constrained Markov Decision Processes, Discounted continuous-time constrained Markov decision processes in Polish spaces, Queuing with future information, Zero-sum constrained stochastic games with independent state processes, Constrained cost-coupled stochastic games with independent state processes, Adaptive control of constrained Markov chains: Criteria and policies, Sensitivity of constrained Markov decision processes, Constrained denumerable state non-stationary MDPs with expected total reward criterion, Notes on average Markov decision processes with a minimum-variance criterion, Sample path optimality for a Markov optimization problem, Optimality of Mixed Policies for Average Continuous-Time Markov Decision Processes with Constraints, OPTIMAL MIXING OF MARKOV DECISION RULES FOR MDP CONTROL, Markov Decision Processes with Asymptotic Average Failure Rate Constraint