Robustness to Incorrect Priors in Partially Observed Stochastic Control
From MaRDI portal
Abstract: We study the continuity properties of optimal solutions to stochastic control problems with respect to initial probability measures and applications of these to the robustness of optimal control policies applied to systems with incomplete or incorrect priors. It is shown that for single and multi-stage optimal cost problems, continuity and robustness cannot be established under weak convergence or Wasserstein convergence in general, but that the optimal cost is continuous in the priors under the convergence in total variation under mild conditions. By imposing further assumptions on the measurement models, robustness and continuity also hold under weak convergence of priors. We thus obtain robustness results and bounds on the mismatch error that occurs due to the application of a control policy which is designed for an incorrectly estimated prior in terms of a distance measure between the true prior and the incorrect one. Positive and negative practical implications of these results in empirical learning for stochastic control will be presented, where almost surely weak convergence of i.i.d. empirical measures occurs but stronger notions of convergence, such as total variation convergence, in general, do not.
Recommendations
- Robustness to incorrect priors and controlled filter stability in partially observed stochastic control
- Robustness to incorrect models and data-driven learning in average-cost optimal stochastic control
- Robustness to incorrect system models in stochastic control
- Robustness of stochastic-parameter control and estimation schemes
- On stochastic observability under incomplete a priori data
- Robust optimal control on imperfect measurements of dynamic systems states
- Pathwise stochastic control with applications to robust filtering
- Partially observable stochastic optimal control
- scientific article; zbMATH DE number 1394785
Cites work
- \(H^ \infty\)-optimal control and related minimax design problems. A dynamic game approach.
- A risk-sensitive maximum principle: the case of imperfect state observation
- Adaptive Markov control processes
- Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal
- Connections between stochastic control and dynamic games
- Convergence of Dynamic Programming Models
- Discrete time nonlinear filters with informative observations are stable
- Dynamic programming subject to total variation distance ambiguity
- Empirical Processes, Typical Sequences, and Coordinated Actions in Standard Borel Spaces
- Entropy bounds on Bayesian learning
- Exponential stability in discrete-time filtering for non-ergodic signal.
- Exponential stability of discrete-time filters for bounded observation noise
- Feedback and optimal sensitivity: Model reference transformations, multiplicative seminorms, and approximate inverses
- Forward-backward stochastic differential games and stochastic control under model uncertainty
- Functional Properties of Minimum Mean-Square Error and Mutual Information
- scientific article; zbMATH DE number 1001726 (Why is no real title available?)
- scientific article; zbMATH DE number 3870398 (Why is no real title available?)
- scientific article; zbMATH DE number 3906790 (Why is no real title available?)
- scientific article; zbMATH DE number 3664132 (Why is no real title available?)
- scientific article; zbMATH DE number 48436 (Why is no real title available?)
- scientific article; zbMATH DE number 3563431 (Why is no real title available?)
- scientific article; zbMATH DE number 1324455 (Why is no real title available?)
- scientific article; zbMATH DE number 1325008 (Why is no real title available?)
- scientific article; zbMATH DE number 1022658 (Why is no real title available?)
- scientific article; zbMATH DE number 1391397 (Why is no real title available?)
- scientific article; zbMATH DE number 3245885 (Why is no real title available?)
- scientific article; zbMATH DE number 3274494 (Why is no real title available?)
- Incomplete information in Markovian decision models
- Intrinsic methods in filter stability
- Markov chains and invariant probabilities
- Minimax optimal control of stochastic uncertain systems with relative entropy constraints
- Near optimality of quantized policies in stochastic control under weak continuity conditions
- On the existence of optimal policies for a class of static and sequential dynamic teams
- Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games
- Optimization and convergence of observation channels in stochastic control
- Partially observable total-cost Markov decision processes with weakly continuous transition probabilities
- Real Analysis and Probability
- Reduction of a Controlled Markov Model with Incomplete Data to a Problem with Complete Information in the Case of Borel State and Control Space
- Robust properties of risk-sensitive control
- Robust sensitivity analysis for stochastic systems
- Stochastic optimal control. The discrete time case
- Stochastic Uncertain Systems Subject to Relative Entropy Constraints: Induced Norms and Monotonicity Properties of Minimax Games
- The universal Glivenko-Cantelli property
- Uniform and universal Glivenko-Cantelli classes
- Uniform Central Limit Theorems
- Uniformity in weak convergence
Cited in
(12)- A robustness result for stochastic control
- Convex analytic method revisited: further optimality results and performance of deterministic policies in average cost stochastic control
- Robustness to incorrect models and data-driven learning in average-cost optimal stochastic control
- Regularized stochastic team problems
- Robustness to incorrect priors and controlled filter stability in partially observed stochastic control
- Robustness to incorrect system models in stochastic control
- Robustness to approximations and model learning in MDPs and POMDPs
- Continuity Properties of Value Functions in Information Structures for Zero-Sum and General Games and Stochastic Teams
- Q-learning in regularized mean-field games
- On Borkar and Young relaxed control topologies and continuous dependence of invariant measures on control policy
- Average cost optimality of partially observed MDPs: contraction of nonlinear filters and existence of optimal solutions and approximations
- Another look at partially observed optimal stochastic control: existence, ergodicity, and approximations without belief-reduction
This page was built for publication: Robustness to Incorrect Priors in Partially Observed Stochastic Control
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5232210)
