Dynamic programming subject to total variation distance ambiguity

DOI10.1137/140955707zbMATH Open1327.90363arXiv1402.1009OpenAlexW1584593412MaRDI QIDQ5502181FDOQ5502181

I. Tzortzis, Charalambos D. Charalambous, Themistoklis Charalambous

Publication date: 18 August 2015

Published in: SIAM Journal on Control and Optimization (Search for Journal in Brave)

Abstract: The aim of this paper is to address optimality of stochastic control strategies via dynamic programming subject to total variation distance ambiguity on the conditional distribution of the controlled process. We formulate the stochastic control problem using minimax theory, in which the control minimizes the pay-off while the conditional distribution, from the total variation distance set, maximizes it. First, we investigate the maximization of a linear functional on the space of probability measures on abstract spaces, among those probability measures which are within a total variation distance from a nominal probability measure, and then we give the maximizing probability measure in closed form. Second, we utilize the solution of the maximization to solve minimax stochastic control with deterministic control strategies, under a Markovian and a non-Markovian assumption, on the conditional distributions of the controlled process. The results of this part include: 1) Minimax optimization subject to total variation distance ambiguity constraint; 2) new dynamic programming recursions, which involve the oscillator seminorm of the value function, in addition to the standard terms; 3) new infinite horizon discounted dynamic programming equation, the associated contractive property, and a new policy iteration algorithm. Finally, we provide illustrative examples for both the finite and infinite horizon cases. For the infinite horizon case we invoke the new policy iteration algorithm to compute the optimal strategies.

Full work available at URL: https://arxiv.org/abs/1402.1009

Recommendations

• Infinite Horizon Average Cost Dynamic Programming Subject to Total Variation Distance Ambiguity
• Optimal control of uncertain stochastic systems subject to total variation distance uncertainty
• Robust Dynamic Programming
• Minimax control for discrete-time time-varying stochastic systems
• scientific article

zbMATH Keywords

dynamic programmingminimaxstochastic controltotal variational distance

Mathematics Subject Classification ID

Dynamic programming (90C39) Optimal stochastic control (93E20) Existence of solutions for minimax problems (49J35)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Stochastic optimal control. The discrete time case
• Dynamic programming and optimal control. Vol. 1.
• Title not available (Why is that?)
• \(H^ \infty\)-optimal control and related minimax design problems. A dynamic game approach.
• Connections between stochastic control and dynamic games
• Title not available (Why is that?)
• Title not available (Why is that?)
• Minimum principle for partially observable nonlinear risk-sensitive control problems using measure-valued decompositions
• A risk-sensitive maximum principle: the case of imperfect state observation
• Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems
• A Finite-Dimensional Risk-Sensitive Control Problem
• On the optimal control of stochastic systems with an exponential-of- integral performance index
• Minimax optimal control of stochastic uncertain systems with relative entropy constraints
• Finite horizon minimax optimal control of stochastic partially observed time varying uncertain systems
• Stochastic Uncertain Systems Subject to Relative Entropy Constraints: Induced Norms and Monotonicity Properties of Minimax Games
• Extremum Problems With Total Variation Distance and Their Applications
• Optimal Control of Uncertain Stochastic Systems Subject to Total Variation Distance Uncertainty

Cited In (5)

• Distributional Robustness in Minimax Linear Quadratic Control with Wasserstein Distance
• Robustness to Incorrect Priors in Partially Observed Stochastic Control
• Robust Markov Decision Processes with Data-Driven, Distance-Based Ambiguity Sets
• Infinite Horizon Average Cost Dynamic Programming Subject to Total Variation Distance Ambiguity
• Robustness to Incorrect System Models in Stochastic Control

This page was built for publication: Dynamic programming subject to total variation distance ambiguity

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5502181)