Lipschitz continuity of value functions in Markovian decision processes

DOI10.1007/S00186-005-0438-1MaRDI QIDQ814878zbMATH OpenOpenAlexFDO

Authors K. Hinderer

Publication date 8 February 2006

Published in Mathematical Methods of Operations Research (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1007/s00186-005-0438-1

zbMATH Keywords

Lipschitz continuityMarkovian decision processesapproximate solution of MDP's by discretization

Mathematics Subject Classification ID

Markov and semi-Markov decision processes (90C40) Nonlinear spectral theory, nonlinear eigenvalue problems (47J10)

Recommendations

• Lipschitz continuous dynamic programming with discount
• Lipschitz continuous dynamic programming with discount II
• Lipschitz continuity of the value function in optimal control
• Lipschitz continuous policy functions for strongly concave optimization problems
• Sensitivity of constrained Markov decision processes

Cited in

(19)

• Lipschitz recursive equilibrium with a minimal state space and heterogeneous agents
• Markov decision processes approximation with coupled dynamics via Markov deterministic control systems
• Approximation of Markov decision processes with general state space
• A stability result for linear Markovian stochastic optimization problems
• Computable approximations for average Markov decision processes in continuous time
• Robustness and sample complexity of model-based MARL for general-sum Markov games
• Policy gradient in Lipschitz Markov decision processes
• Lipschitz continuity and semiconcavity properties of the value function of a stochastic control problem
• Lipschitz continuous dynamic programming with discount
• scientific article; zbMATH DE number 786222 (Why is no real title available?)
• A constructive geometrical approach to the uniqueness of Markov stationary equilibrium in stochastic games of intergenerational altruism
• Continuity of the value of competitive Markov decision processes
• scientific article; zbMATH DE number 7625165 (Why is no real title available?)
• Generalized envelope theorems: applications to dynamic programming
• Mean-field controls with Q-learning for cooperative MARL: convergence and complexity analysis
• Lipschitz continuous dynamic programming with discount II
• An unbounded Berge's minimum theorem with applications to discounted Markov decision processes
• First-order sensitivity of the optimal value in a Markov decision model with respect to deviations in the transition probability function
• Stochastic approximations of constrained discounted Markov decision processes

This page was built for publication: Lipschitz continuity of value functions in Markovian decision processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q814878)