Risk-sensitive Markov control processes
From MaRDI portal
Abstract: We introduce a general framework for measuring risk in the context of Markov control processes with risk maps on general Borel spaces that generalize known concepts of risk measures in mathematical finance, operations research and behavioral economics. Within the framework, applying weighted norm spaces to incorporate also unbounded costs, we study two types of infinite-horizon risk-sensitive criteria, discounted total risk and average risk, and solve the associated optimization problems by dynamic programming. For the discounted case, we propose a new discount scheme, which is different from the conventional form but consistent with the existing literature, while for the average risk criterion, we state Lyapunov-like stability conditions that generalize known conditions for Markov chains to ensure the existence of solutions to the optimality equation.
The authors propose a generalized definition of convex risk measures (CRMs) to include the measures considered in behavioral economics and they apply a constructive approach which maintains the Markov property assuring existence of stationary optimal policies for two infinite-horizon objectives, namely discounted total risk and average risk. The definition is extended onto Markovian temporal structures called risk maps which are discussed in the Markov control processes context by adding control parameters. The two types of infinite-horizon objectives are optimized via dynamic programming techniques. The results are compared with classical Markov control processes and with entropic maps. As an example, the authors discuss a one-dimensional simple linear model with mean-semideviation and prove that the proposed sufficient conditions for average risk are satisfied. The paper is difficult to read and the practical implications of the presented results are not clear.
Recommendations
- Risk-sensitive control of continuous time Markov chains
- Risk sensitive control of Markov processes in countable state space
- On risk sensitive control of regular step Markov processes
- scientific article; zbMATH DE number 1775015
- Risk sensitive impulse control of non-Markovian processes
- Risk-Sensitive Control of Discrete-Time Markov Processes with Infinite Horizon
- On risk-sensitive ergodic impulsive control of Markov processes
Cited in
(55)- Risk measurement and risk-averse control of partially observable discrete-time Markov systems
- Optimal dividend payout model with risk sensitive preferences
- Variance-constrained actor-critic algorithms for discounted and average reward MDPs
- Probabilistically distorted risk-sensitive infinite-horizon dynamic programming
- Risk sensitive control of finite state Markov chains in discrete time, with applications to portfolio management
- A discounted approach in communicating average Markov decision chains under risk-aversion
- Long-run risk sensitive dyadic impulse control
- Risk-averse autonomous systems: a brief history and recent developments from the perspective of optimal control
- Process-based risk measures and risk-averse control of discrete-time systems
- Discounted approximations in risk-sensitive average Markov cost chains with finite state space
- Markov decision processes with recursive risk measures
- Prospect-theoretic Q-learning
- Mathematical models of risk control for regenerating Markov processes
- Discounted approximations to the risk-sensitive average cost in finite Markov chains
- Risk-sensitive probability for Markov chains
- Dynamic programming with value convexity
- Continuous-time Markov decision processes under the risk-sensitive first passage discounted cost criterion
- Markov risk mappings and risk-sensitive optimal prediction
- On modeling risk in Markov decision processes.
- Long run risk sensitive portfolio with general factors
- Computational methods for risk-averse undiscounted transient Markov models
- Markov decision processes with iterated coherent risk measures
- Risk-sensitive optimal control of hidden Markov models: structural results
- On risk sensitive control of regular step Markov processes
- The vanishing discount approach in a class of zero-sum finite games with risk-sensitive average criterion
- A new approach to risk sensitivity
- Risk-averse learning by temporal difference methods with Markov risk measures
- Risk-sensitive Markov decision problems under model uncertainty: finite time horizon case
- Discrete-time risk-aware optimal switching with non-adapted costs
- Risk-Sensitive Reinforcement Learning via Policy Gradient Search
- More risk-sensitive Markov decision processes
- Vanishing discount approximations in controlled Markov chains with risk-sensitive average criterion
- Risk-sensitive semi-Markov decision processes with general utilities and multiple criteria
- Characterization of the optimal risk-sensitive average cost in denumerable Markov decision chains
- Risk-averse control of undiscounted transient Markov models
- Risk-sensitive reinforcement learning
- Risk-Sensitive Optimal Control for Markov Decision Processes with Monotone Cost
- Ergodic risk-sensitive control of Markov processes on countable state space revisited
- Distributionally Robust Markov Decision Processes and Their Connection to Risk Measures
- Risk-Sensitive Average Optimality for Discrete-Time Markov Decision Processes
- Contractive approximations in average Markov decision chains driven by a risk-seeking controller
- Risk filtering and risk-averse control of Markovian systems subject to model uncertainty
- A dynamic analytic method for risk-aware controlled martingale problems
- Matrix-Analytic Methods for Solving Poisson’s Equation with Applications to Markov Chains of GI/G/1-Type
- Discrete‐time risk sensitive portfolio optimization with proportional transaction costs
- Duality between large deviation control and risk-sensitive control for Markov decision processes
- Risk-sensitivity vanishing limit for controlled Markov processes
- Risk-sensitive Markov decision under risk constraints with coherent risk measures
- Markov decision processes with risk-sensitive criteria: an overview
- Risk-sensitive average Markov decision processes in general spaces
- Error analysis for approximate CVaR-optimal control with a maximum cost
- Dynamic risk control in multidimensional Markov models
- Robust optimal control using conditional risk mappings in infinite horizon
- Yet another distributional Bellman equation
- Zero-sum risk-sensitive stochastic games
This page was built for publication: Risk-sensitive Markov control processes
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2873849)
