Convergence of policy iteration for entropy-regularized stochastic control problems

DOI10.1137/24M1638744MaRDI QIDQ7009916zbMATH OpenFDO

Authors Yu-Jui Huang, Zhenhua Wang, Zhou Zhou

Publication date 19 March 2025

Published in SIAM Journal on Control and Optimization (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1137/24M1638744

zbMATH Keywords

entropy regularizationexploratory Hamilton-Jacobi-Bellman equationspolicy iteration algorithmreinforcement learning

Mathematics Subject Classification ID

Hamilton-Jacobi equations in optimal control and differential games (49L12) Stochastic ordinary differential equations (aspects of stochastic analysis) (60H10) Optimal stochastic control (93E20) Measures of information, entropy (94A17)

Cites work

• A Mathematical Theory of Communication
• A modified MSA for stochastic control problems
• Continuous‐time mean–variance portfolio selection: A reinforcement learning framework
• Elliptic partial differential equations of second order
• Entropy Regularization for Mean Field Games with Learning
• Exploratory HJB equations and their convergence
• Exploratory LQG mean field games with entropy regularization
• Exponential convergence and stability of Howard's policy improvement algorithm for controlled diffusions
• scientific article; zbMATH DE number 51724 (Why is no real title available?)
• scientific article; zbMATH DE number 1161554 (Why is no real title available?)
• scientific article; zbMATH DE number 1181255 (Why is no real title available?)
• scientific article; zbMATH DE number 7307478 (Why is no real title available?)
• Information Theory and Statistical Mechanics
• Learning equilibrium mean‐variance strategy
• On the convergence of policy iteration for controlled diffusions
• On the policy improvement algorithm in continuous time
• Policy iteration for the deterministic control problems -- a viscosity approach
• Policy iterations for reinforcement learning problems in continuous time and space -- fundamental theory and methods
• Regularity and stability of feedback relaxed controls

Cited in

(5)

• Policy iteration for nonconvex viscous Hamilton-Jacobi equations
• Continuous-time optimal investment with portfolio constraints: a reinforcement learning approach
• Convergence analysis for entropy-regularized control problems: a probabilistic approach
• Feedback Cycles in Exploratory Equilibria
• Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence

This page was built for publication: Convergence of policy iteration for entropy-regularized stochastic control problems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q7009916)