Robustness to incorrect models and data-driven learning in average-cost optimal stochastic control
From MaRDI portal
(Redirected from Publication:2116649)
Abstract: We study continuity and robustness properties of infinite-horizon average expected cost problems with respect to (controlled) transition kernels, and applications of these results to the problem of robustness of control policies designed for approximate models applied to actual systems. We show that sufficient conditions presented in the literature for discounted-cost problems are in general not sufficient to ensure robustness for average-cost problems. However, we show that the average optimal cost is continuous in the convergences of controlled transition kernel models where convergence of models entails (i) continuous weak convergence in state and actions, and (ii) continuous setwise convergence in the actions for every fixed state variable, in addition to either uniform ergodicity or some regularity conditions. We establish that the mismatch error due to the application of a control policy designed for an incorrectly estimated model to the true model decreases to zero as the incorrect model approaches the true model under the stated convergence criteria. Our findings significantly relax related studies in the literature which have primarily considered the more restrictive total variation convergence criteria. Applications to robustness to models estimated through empirical data (where almost sure weak convergence criterion typically holds, but stronger criteria do not) are studied and conditions for asymptotic robustness to data-driven learning are established.
Recommendations
- Robustness to incorrect system models in stochastic control
- Robustness to approximations and model learning in MDPs and POMDPs
- Robustness to Incorrect Priors in Partially Observed Stochastic Control
- Stochastic Control with Imperfect Models
- Infinite Horizon Average Cost Dynamic Programming Subject to Total Variation Distance Ambiguity
Cites work
- scientific article; zbMATH DE number 3870398 (Why is no real title available?)
- scientific article; zbMATH DE number 3906790 (Why is no real title available?)
- scientific article; zbMATH DE number 46153 (Why is no real title available?)
- scientific article; zbMATH DE number 48436 (Why is no real title available?)
- scientific article; zbMATH DE number 1391397 (Why is no real title available?)
- scientific article; zbMATH DE number 3245885 (Why is no real title available?)
- A Universal Empirical Dynamic Programming Algorithm for Continuous State MDPs
- A Useful Convergence Theorem for Probability Distributions
- Adapted Wasserstein distances and stability in mathematical finance
- Adaptive Markov control processes
- All adapted topologies are equal
- Ambiguous chance constrained problems and robust optimization
- Analyticity, Convergence, and Convergence Rate of Recursive Maximum-Likelihood Estimation in Hidden Markov Models
- Approximation of average cost Markov decision processes using empirical distributions and concentration inequalities
- Bayesian nonparametrics
- Connections between stochastic control and dynamic games
- Convergence analysis for distributionally robust optimization and equilibrium problems
- Convergence of Dynamic Programming Models
- Discrete Time Stochastic Adaptive Control
- Dynamic programming and stochastic control
- Empirical Processes, Typical Sequences, and Coordinated Actions in Standard Borel Spaces
- Empirical dynamic programming
- Exponential filter stability via Dobrushin's coefficient
- Minimax optimal control of stochastic uncertain systems with relative entropy constraints
- On robustness of discrete time optimal filters
- Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games
- Real Analysis and Probability
- Recurrence conditions for Markov decision processes with Borel state space: A survey
- Robust H∞ infinity control in the presence of stochastic uncertainty
- Robust H/sub infinity / control for linear systems with norm-bounded time-varying uncertainty
- Robust properties of risk-sensitive control
- Robustness to incorrect system models in stochastic control
- Sequential decisions under uncertainty and the maximum theorem
- Stability of a 4th-order curvature condition arising in optimal transport theory
- Stability of optimal filter higher-order derivatives
- Stochastic Control with Imperfect Models
- The universal Glivenko-Cantelli property
- Uniform Central Limit Theorems
- Uniform and universal Glivenko-Cantelli classes
Cited in
(3)
This page was built for publication: Robustness to incorrect models and data-driven learning in average-cost optimal stochastic control
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2116649)
