Robust Q-learning algorithm for Markov decision processes under Wasserstein uncertainty

DOI10.1016/j.automatica.2024.111825MaRDI QIDQ6605954zbMATH OpenFDO

Authors Ariel Neufeld, Julian Sester

Publication date 16 September 2024

Published in Automatica (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1016/j.automatica.2024.111825

zbMATH Keywords

\(Q\)-learningdistributionally robust optimizationMarkov decision processreinforcement learningWasserstein uncertainty

Mathematics Subject Classification ID

Markov and semi-Markov decision processes (90C40)

Recommendations

• Q-learning for distributionally robust Markov decision processes
• Robust Markov Decision Processes with Data-Driven, Distance-Based Ambiguity Sets
• Distributionally robust Markov decision processes
• Reinforcement learning in robust Markov decision processes
• Minimax Q-learning control for linear systems using the Wasserstein metric

Cites work

• \({\mathcal Q}\)-learning
• A new approach to variable metric algorithms
• A Stochastic Approximation Method
• Computational aspects of robust optimized certainty equivalents and option pricing
• Conditional Risk Mappings
• Conditioning of Quasi-Newton Methods for Function Minimization
• Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations
• Data-driven risk-averse stochastic optimization with Wasserstein metric
• Distributionally Robust Control of Constrained Stochastic Systems
• Distributionally Robust Convex Optimization
• Distributionally robust Markov decision processes
• Distributionally robust optimization for sequential decision-making
• Double deep Q-learning for optimal execution
• scientific article; zbMATH DE number 3121164 (Why is no real title available?)
• Machine learning in finance. From theory to practice
• Markov decision processes under model uncertainty
• Markov decision processes with applications to finance.
• Model-free Q-learning designs for linear discrete-time zero-sum games with application to H^ control
• Monge-Kantorovich transportation problem and optimal couplings
• On the Convergence of Stochastic Iterative Dynamic Programming Algorithms
• Q-learning for distributionally robust Markov decision processes
• Quantifying distributional model risk via optimal transport
• Rectangular sets of probability measures
• Reinforcement learning. An introduction
• Robust Control of Markov Decision Processes with Uncertain Transition Matrices
• Robust Dynamic Programming
• Robust Markov Decision Processes
• Robust Markov Decision Processes: Beyond Rectangularity
• Robust MDPs with k-rectangular uncertainty
• Robust optimal control using conditional risk mappings in infinite horizon
• Some better bounds on the variance with applications
• Sur les équations algébriques ayant toutes leurs racines réelles.
• The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations
• Wasserstein Distributionally Robust Stochastic Control: A Data-Driven Approach

Cited in

(3)

• Bounding the difference between the values of robust and non-robust Markov decision problems
• Sequential decision-making under uncertainty: a robust MDPs review
• Existence of viscosity solutions to abstract Cauchy problems via nonlinear semigroups

This page was built for publication: Robust \(Q\)-learning algorithm for Markov decision processes under Wasserstein uncertainty

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6605954)