First-order sensitivity of the optimal value in a Markov decision model with respect to deviations in the transition probability function
From MaRDI portal
Publication:2216181
Abstract: Markov decision models (MDM) used in practical applications are most often less complex than the underlying `true' MDM. The reduction of model complexity is performed for several reasons. However, it is obviously of interest to know what kind of model reduction is reasonable (in regard to the optimal value) and what kind is not. In this article we propose a way how to address this question. We introduce a sort of derivative of the optimal value as a function of the transition probabilities, which can be used to measure the (first-order) sensitivity of the optimal value w.r.t. changes in the transition probabilities. `Differentiability' is obtained for a fairly broad class of MDMs, and the `derivative' is specified explicitly. Our theoretical findings are illustrated by means of optimization problems in inventory control and mathematical finance.
Recommendations
- scientific article; zbMATH DE number 3987072
- Sensitivity analysis in Markov decision processes with uncertain reward parameters
- Sensitivity analysis in discounted Markovian decision problems
- How Does the Value Function of a Markov Decision Process Depend on the Transition Probabilities?
- Sensitivity of constrained Markov decision processes
Cites work
- Bounds for the regret loss in dynamic programming under adaptive control
- Calculation of the Wasserstein Distance Between Probability Distributions on the Line
- Continuous-time stochastic control and optimization with financial applications
- Domains of weak continuity of statistical functionals with a view toward robust statistics
- Generalized quantiles as risk measures
- How Does the Value Function of a Markov Decision Process Depend on the Transition Probabilities?
- scientific article; zbMATH DE number 3122046 (Why is no real title available?)
- scientific article; zbMATH DE number 3120402 (Why is no real title available?)
- scientific article; zbMATH DE number 3134565 (Why is no real title available?)
- scientific article; zbMATH DE number 49698 (Why is no real title available?)
- scientific article; zbMATH DE number 4128239 (Why is no real title available?)
- scientific article; zbMATH DE number 700091 (Why is no real title available?)
- scientific article; zbMATH DE number 1022519 (Why is no real title available?)
- scientific article; zbMATH DE number 1909499 (Why is no real title available?)
- scientific article; zbMATH DE number 3320878 (Why is no real title available?)
- scientific article; zbMATH DE number 3386359 (Why is no real title available?)
- Integral Probability Metrics and Their Generating Classes of Functions
- Lipschitz continuity of value functions in Markovian decision processes
- Markov decision processes with applications to finance.
- Markov programming by successive approximations with respect to weighted supremum norms
- Nonsmooth analysis
- On concepts of directional differentiability
- Perturbation theory for Markov reward processes with applications to queueing systems
- Perturbation theory for unbounded Markov reward processes with applications to queueing
- Probability Metrics
- Qualitative and infinitesimal robustness of tail-dependent statistical functionals
- Rate of convergence of an empirical regression method for solving generalized backward stochastic differential equations
- Real Analysis and Probability
- Statistical inference for expectile-based risk measures
- Stochastic finance. An introduction in discrete time.
- The theory of differentiation in linear topological spaces
- Von Mises calculus for statistical functionals
Cited in
(4)- A concept of copula robustness and its applications in quantitative risk management
- Sensitivity analysis in Markov decision processes with uncertain reward parameters
- How Does the Value Function of a Markov Decision Process Depend on the Transition Probabilities?
- An Integrated Transportation Distance between Kernels and Approximate Dynamic Risk Evaluation in Markov Systems
This page was built for publication: First-order sensitivity of the optimal value in a Markov decision model with respect to deviations in the transition probability function
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2216181)
