Near optimality of quantized policies in stochastic control under weak continuity conditions

DOI10.1016/J.JMAA.2015.10.008zbMATH Open1323.93078arXiv1410.6985OpenAlexW2963272571MaRDI QIDQ892326FDOQ892326

Authors: Naci Saldi, Serdar Yüksel, Tamás Linder 

Publication date: 18 November 2015

Published in: Journal of Mathematical Analysis and Applications (Search for Journal in Brave)

Abstract: This paper studies the approximation of optimal control policies by quantized (discretized) policies for a very general class of Markov decision processes (MDPs). The problem is motivated by applications in networked control systems, computational methods for MDPs, and learning algorithms for MDPs. We consider the finite-action approximation of stationary policies for a discrete-time Markov decision process with discounted and average costs under a weak continuity assumption on the transition probability, which is a significant relaxation of conditions required in earlier literature. The discretization is constructive, and quantized policies are shown to approximate optimal deterministic stationary policies with arbitrary precision. The results are applied to the fully observed reduction of a partially observed Markov decision process, where weak continuity is a much more reasonable assumption than more stringent conditions such as strong continuity or continuity in total variation.

Full work available at URL: https://arxiv.org/abs/1410.6985

Recommendations

• Asymptotic Optimality and Rates of Convergence of Quantized Stationary Policies in Stochastic Control
• Asymptotic optimality and rates of convergence of quantized stationary policies in continuous-time Markov decision processes
• Finite approximations in discrete-time stochastic control. Quantized models and asymptotic optimality
• Asymptotic optimality of quantized stationary policies in continuous-time Markov decision processes with Polish spaces
• Stochastic Near-Optimal Controls: Necessary and Sufficient Conditions for Near-Optimality
• Conditions for optimality of Naïve quantized finite horizon control
• Finite Model Approximations and Asymptotic Optimality of Quantized Policies in Decentralized Stochastic Control
• A revisit to stochastic near-optimal controls: the critical case

zbMATH Keywords

approximationquantizationstochastic controlpartially observed Markov decision processes

Mathematics Subject Classification ID

Markov and semi-Markov decision processes (90C40) Optimal stochastic control (93E20)

Cites Work

• Measure theory. Vol. I and II
• Title not available (Why is that?)
• Stochastic optimal control. The discrete time case
• Adaptive Markov control processes
• Stochastic networked control systems. Stabilization and optimization under information constraints
• Nonparametric Estimation of Conditional Distributions
• Average cost Markov control processes with weighted norms: existence of canonical policies
• Finite linear programming approximations of constrained discounted Markov decision processes
• Asymptotic Optimality and Rates of Convergence of Quantized Stationary Policies in Stochastic Control
• Convergence of Dynamic Programming Models
• Convergence of discretization procedures in dynamic programming
• Title not available (Why is that?)
• Approximation of Markov decision processes with general state space
• Finite-state approximations for denumerable-state infinite-horizon discounted Markov decision processes
• Markov chains and invariant probabilities
• Average cost Markov decision processes with weakly continuous transition probabilities
• Incomplete information in Markovian decision models
• Reduction of a Controlled Markov Model with Incomplete Data to a Problem with Complete Information in the Case of Borel State and Control Space
• The average cost optimality equation: a fixed point approach
• Simulation‐based Uniform Value Function Estimates of Markov Decision Processes
• NEARLY OPTIMAL CONTROL OF NONLINEAR MARKOVIAN SYSTEMS SUBJECT TO WEAK AND STRONG INTERACTIONS
• Nearly optimal control of singularly perturbed Markov decision processes in discrete time

Cited In (11)

• Weak Feller property of non-linear filters
• On Borkar and Young relaxed control topologies and continuous dependence of invariant measures on control policy
• Robustness to incorrect system models in stochastic control
• Robustness to Incorrect Priors in Partially Observed Stochastic Control
• Improved order 1/4 convergence for piecewise constant policy approximation of stochastic control problems
• Finite approximations in discrete-time stochastic control. Quantized models and asymptotic optimality
• Asymptotic optimality of quantized stationary policies in continuous-time Markov decision processes with Polish spaces
• Convex analytic method revisited: further optimality results and performance of deterministic policies in average cost stochastic control
• Robustness to approximations and model learning in MDPs and POMDPs
• Asymptotic optimality and rates of convergence of quantized stationary policies in continuous-time Markov decision processes
• Continuity of cost in Borkar control topology and implications on discrete space and time approximations for controlled diffusions under several criteria

This page was built for publication: Near optimality of quantized policies in stochastic control under weak continuity conditions

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q892326)