Reinforcement learning with dynamic convex risk measures

DOI10.1111/MAFI.12388arXiv2112.13414OpenAlexW4225482307MaRDI QIDQ6196296FDOQ6196296

Authors: Sebastian Jaimungal 

Publication date: 14 March 2024

Published in: Mathematical Finance (Search for Journal in Brave)

Abstract: We develop an approach for solving time-consistent risk-sensitive stochastic optimization problems using model-free reinforcement learning (RL). Specifically, we assume agents assess the risk of a sequence of random variables using dynamic convex risk measures. We employ a time-consistent dynamic programming principle to determine the value of a particular policy, and develop policy gradient update rules that aid in obtaining optimal policies. We further develop an actor-critic style algorithm using neural networks to optimize over policies. Finally, we demonstrate the performance and flexibility of our approach by applying it to three optimization problems: statistical arbitrage trading strategies, financial hedging, and obstacle avoidance robot control.

Full work available at URL: https://arxiv.org/abs/2112.13414

zbMATH Keywords

time-consistencyreinforcement learningactor-critic algorithmrobot controldynamic risk measurestrading strategiespolicy gradientfinancial hedging

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Statistical methods; risk measures (91G70) Optimal stochastic control (93E20)

Cites Work

• A Closed-Form Solution for Options with Stochastic Volatility with Applications to Bond and Currency Options
• Coherent measures of risk
• DISTRIBUTION‐INVARIANT RISK MEASURES, INFORMATION, AND DYNAMIC CONSISTENCY
• Title not available (Why is that?)
• Envelope Theorems for Arbitrary Choice Sets
• Stochastic finance. An introduction in discrete time
• Convex measures of risk and trading constraints
• Conditional and dynamic convex risk measures
• Title not available (Why is that?)
• Dynamic risk measures
• Dynamic assessment indices
• Approximation by superpositions of a sigmoidal function
• Title not available (Why is that?)
• Lectures on stochastic programming. Modeling and theory.
• Title not available (Why is that?)
• Dual representation of minimal supersolutions of convex BSDEs
• Percentile Optimization for Markov Decision Processes with Parameter Uncertainty
• Dynamic coherent risk measures
• Risk-averse dynamic programming for Markov decision processes
• Risk Measures and Comonotonicity: A Review
• Approximate Integration of Stochastic Differential Equations
• Sequential Decision Making With Coherent Risk
• Markov decision processes with iterated coherent risk measures
• Markov decision processes with recursive risk measures
• Approximate Value Iteration for Risk-Aware Markov Decision Processes
• Solving high-dimensional partial differential equations using deep learning
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Title not available (Why is that?)
• Title not available (Why is that?)
• Reinforcement learning and stochastic optimisation
• Minimizing spectral risk measures applied to Markov decision processes
• Risk-Constrained Reinforcement Learning with Percentile Risk Criteria
• Risk-Sensitive Reinforcement Learning
• Deep Q-Learning for Nash Equilibria: Nash-DQN
• Deep learning volatility: a deep neural network perspective on pricing and calibration in (rough) volatility models
• Policy Gradient Methods for the Noisy Linear Quadratic Regulator over a Finite Horizon
• Distributionally Robust Markov Decision Processes and Their Connection to Risk Measures

Cited In (1)

• Markov decision processes with risk-sensitive criteria: an overview

This page was built for publication: Reinforcement learning with dynamic convex risk measures

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6196296)