A unified approach to Markov decision problems and performance sensitivity analysis with discounted and average criteria: multichain cases

DOI10.1016/J.AUTOMATICA.2004.05.003MaRDI QIDQ705478zbMATH OpenOpenAlexFDO

Authors Xianping Guo, Cao, Xiren

Publication date 31 January 2005

Published in Automatica (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1016/j.automatica.2004.05.003

zbMATH Keywords

PotentialsReinforcement learningPerformance sensitivityPerturbation analysisPolicy iteration

Mathematics Subject Classification ID

Sensitivity (robustness) (93B35) Markov and semi-Markov decision processes (90C40) Optimal stochastic control (93E20)

Recommendations

• A unified approach to Markov decision problems and performance sensitivity analysis
• scientific article; zbMATH DE number 1944273
• A unified approach to time-aggregated Markov decision processes
• scientific article; zbMATH DE number 2159039
• From perturbation analysis to Markov decision processes and reinforcement learning

Cites work

• scientific article; zbMATH DE number 3148886 (Why is no real title available?)
• scientific article; zbMATH DE number 51708 (Why is no real title available?)
• scientific article; zbMATH DE number 700091 (Why is no real title available?)
• scientific article; zbMATH DE number 1753152 (Why is no real title available?)
• scientific article; zbMATH DE number 2189770 (Why is no real title available?)
• A Brouwer fixed-point mapping approach to communicating Markov decision processes
• A Fixed Point Approach to Undiscounted Markov Renewal Programs
• A note on policy algorithms for discounted Markov decision problems
• A unified approach to Markov decision problems and performance sensitivity analysis
• CONVERGENCE OF SIMULATION-BASED POLICY ITERATION
• Discrete Dynamic Programming
• Foolproof convergence in multichain policy iteration
• From perturbation analysis to Markov decision processes and reinforcement learning
• Limiting average criteria for nonstationary Markov decision processes
• Minimax control for discrete-time time-varying stochastic systems
• On Finding Optimal Policies in Discrete Dynamic Programming with No Discounting
• Perturbation realization, potentials, and sensitivity analysis of Markov processes
• Realization probabilities. The dynamics of queuing systems
• Simulation-based optimization of Markov reward processes
• Single sample path-based optimization of Markov chains
• The Maclaurin series for performance functions of Markov chains
• The relations among potentials, perturbation analysis, and Markov decision processes

Cited in

(12)

• A unified approach to time-aggregated Markov decision processes
• Bias optimality for multichain continuous-time Markov decision processes
• Performance analysis of AIMD mechanisms over a multi-state Markovian path
• Temporal difference-based policy iteration for optimal control of stochastic systems
• Unified NDP method based on TD(0) learning for both average and discounted Markov decision processes
• Basic ideas for event-based optimization of Markov systems
• scientific article; zbMATH DE number 2159039 (Why is no real title available?)
• Continuous-time Markov decision processes with \(n\)th-bias optimality criteria
• scientific article; zbMATH DE number 1944273 (Why is no real title available?)
• Optimization of a special case of continuous-time Markov decision processes with compact action set
• Completion-of-squares: revisited and extended
• The risk probability criterion for discounted continuous-time Markov decision processes

This page was built for publication: A unified approach to Markov decision problems and performance sensitivity analysis with discounted and average criteria: multichain cases

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q705478)