Robustness to incorrect system models in stochastic control
From MaRDI portal
Publication:5111064
Abstract: In stochastic control applications, typically only an ideal model (controlled transition kernel) is assumed and the control design is based on the given model, raising the problem of performance loss due to the mismatch between the assumed model and the actual model. Toward this end, we study continuity properties of discrete-time stochastic control problems with respect to system models (i.e., controlled transition kernels) and robustness of optimal control policies designed for incorrect models applied to the true system. We study both fully observed and partially observed setups under an infinite horizon discounted expected cost criterion. We show that continuity and robustness cannot be established under weak and setwise convergences of transition kernels in general, but that the expected induced cost is robust under total variation. By imposing further assumptions on the measurement models and on the kernel itself (such as continuous convergence), we show that the optimal cost can be made continuous under weak convergence of transition kernels as well. Using these continuity properties, we establish convergence results and error bounds due to mismatch that occurs by the application of a control policy which is designed for an incorrectly estimated system model to a true model, thus establishing positive and negative results on robustness.Compared to the existing literature, we obtain strictly refined robustness results that are applicable even when the incorrect models can be investigated under weak convergence and setwise convergence criteria (with respect to a true model), in addition to the total variation criteria. These entail positive implications on empirical learning in (data-driven) stochastic control since often system models are learned through empirical training data where typically weak convergence criterion applies but stronger convergence criteria do not.
Recommendations
- Robustness to approximations and model learning in MDPs and POMDPs
- Robustness to incorrect models and data-driven learning in average-cost optimal stochastic control
- Robustness to Incorrect Priors in Partially Observed Stochastic Control
- Robustness to incorrect priors and controlled filter stability in partially observed stochastic control
- Stochastic Control with Imperfect Models
Cites work
- scientific article; zbMATH DE number 1001726 (Why is no real title available?)
- scientific article; zbMATH DE number 3870398 (Why is no real title available?)
- scientific article; zbMATH DE number 3906790 (Why is no real title available?)
- scientific article; zbMATH DE number 46153 (Why is no real title available?)
- scientific article; zbMATH DE number 48436 (Why is no real title available?)
- scientific article; zbMATH DE number 3456236 (Why is no real title available?)
- scientific article; zbMATH DE number 3563431 (Why is no real title available?)
- scientific article; zbMATH DE number 1325008 (Why is no real title available?)
- scientific article; zbMATH DE number 765034 (Why is no real title available?)
- scientific article; zbMATH DE number 935561 (Why is no real title available?)
- scientific article; zbMATH DE number 3245885 (Why is no real title available?)
- scientific article; zbMATH DE number 3274494 (Why is no real title available?)
- Accelerating the convergence of value iteration by using partial transition functions
- Ambiguous chance constrained problems and robust optimization
- Average Optimality in Markov Control Processes via Discounted-Cost Problems and Linear Programming
- Bayesian nonparametrics
- Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal
- Connections between stochastic control and dynamic games
- Convergence analysis for distributionally robust optimization and equilibrium problems
- Convergence of Dynamic Programming Models
- Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations
- Dynamic programming subject to total variation distance ambiguity
- Entropy bounds on Bayesian learning
- Finite approximations in discrete-time stochastic control. Quantized models and asymptotic optimality
- Forward-backward stochastic differential games and stochastic control under model uncertainty
- How Does the Value Function of a Markov Decision Process Depend on the Transition Probabilities?
- Markov-Nash equilibria in mean-field games with discounted cost
- Minimax optimal control of stochastic uncertain systems with relative entropy constraints
- Near optimality of quantized policies in stochastic control under weak continuity conditions
- Nonparametric Estimation of Conditional Distributions
- On robustness of discrete time optimal filters
- On the asymptotic optimality of finite approximations to Markov decision processes with Borel spaces
- On the sample complexity of the linear quadratic regulator
- Optimal Approximation Schedules for a Class of Iterative Algorithms, With an Application to Multigrid Value Iteration
- Optimal Transport
- Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games
- Optimization and convergence of observation channels in stochastic control
- Partially observable total-cost Markov decision processes with weakly continuous transition probabilities
- Quantifying distributional model risk via optimal transport
- Random processes for engineers
- Real Analysis and Probability
- Robust H∞ infinity control in the presence of stochastic uncertainty
- Robust Control of Markov Decision Processes with Uncertain Transition Matrices
- Robust Dynamic Programming
- Robust Markov Decision Processes
- Robust properties of risk-sensitive control
- Robust sensitivity analysis for stochastic systems
- Robustness and risk-sensitive filtering
- Robustness to Incorrect Priors in Partially Observed Stochastic Control
- Statistical Methods in Markov Chains
- Stochastic optimal control. The discrete time case
- Weak Feller property of non-linear filters
- White-Noise Representations in Stochastic Realization Theory
- \(H^ \infty\)-optimal control and related minimax design problems. A dynamic game approach.
Cited in
(16)- Robustness to Incorrect Priors in Partially Observed Stochastic Control
- Robustness to incorrect priors and controlled filter stability in partially observed stochastic control
- Robustness to incorrect models and data-driven learning in average-cost optimal stochastic control
- Continuity of discounted values and the structure of optimal policies for <scp>periodic‐review</scp> inventory systems with setup costs
- A robustness result for stochastic control
- Q-learning in regularized mean-field games
- Evaluating the adequacy of models of controlled dynamic systems
- Control plans in models with classification errors
- Average cost optimality of partially observed MDPs: contraction of nonlinear filters and existence of optimal solutions and approximations
- Regularized stochastic team problems
- Model‐system parameter mismatch in GPC control
- Robustness to approximations and model learning in MDPs and POMDPs
- Another look at partially observed optimal stochastic control: existence, ergodicity, and approximations without belief-reduction
- Reinforcement Learning for Linear-Convex Models with Jumps via Stability Analysis of Feedback Controls
- Regularity and stability of feedback relaxed controls
- Geometry of information structures, strategic measures and associated stochastic control topologies
This page was built for publication: Robustness to incorrect system models in stochastic control
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5111064)
