Convergence analysis for entropy-regularized control problems: a probabilistic approach

DOI10.1137/24M1680039MaRDI QIDQ7234359zbMATH OpenFDO

Authors Jin Ma, Gaozhan Wang, Jianfeng Zhang

Publication date 28 April 2026

Published in SIAM Journal on Control and Optimization (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1137/24M1680039

zbMATH Keywords

Bismut-Elworthy-Li formulaentropy regularizationFeynman-Kac formulamodel uncertaintypolicy iteration algorithmrate of convergencereinforcement learning

Mathematics Subject Classification ID

PDEs in connection with control and optimization (35Q93) Applications of stochastic analysis (to PDEs, etc.) (60H30) Stochastic learning and adaptive control (93E35)

Cites work

• A modified MSA for stochastic control problems
• A neural network-based policy iteration algorithm with global \(H^2\)-superlinear convergence for stochastic games on domains
• Continuous‐time mean–variance portfolio selection: A reinforcement learning framework
• Convergence of policy iteration for entropy-regularized stochastic control problems
• Convergence Properties of Policy Iteration
• Entropy annealing for policy mirror descent in continuous time and space
• Entropy Regularization for Mean Field Games with Learning
• Exploratory HJB equations and their convergence
• Exponential convergence and stability of Howard's policy improvement algorithm for controlled diffusions
• Formulae for the derivatives of heat semigroups
• FUNCTIONAL EQUATIONS IN THE THEORY OF DYNAMIC PROGRAMMING. V. POSITIVITY AND QUASI-LINEARITY
• scientific article; zbMATH DE number 3126094 (Why is no real title available?)
• scientific article; zbMATH DE number 3148886 (Why is no real title available?)
• scientific article; zbMATH DE number 3505981 (Why is no real title available?)
• scientific article; zbMATH DE number 947827 (Why is no real title available?)
• scientific article; zbMATH DE number 7307478 (Why is no real title available?)
• Large deviations and the Malliavin calculus
• Linear Convergence of a Policy Gradient Method for Some Finite Horizon Continuous Time Control Problems
• On the convergence of policy iteration for controlled diffusions
• On the Convergence of Policy Iteration in Stationary Dynamic Programming
• On the policy improvement algorithm in continuous time
• Policy iteration for exploratory Hamilton-Jacobi-Bellman equations
• Randomized Optimal Stopping Problem in Continuous time and Reinforcement Learning Algorithm
• Regularity and stability of feedback relaxed controls
• Representation theorems for backward stochastic differential equations
• Some Convergence Results for Howard's Algorithm
• The Malliavin Calculus and Related Topics

Cited in

(3)

• Feedback Cycles in Exploratory Equilibria
• Accuracy of discretely sampled stochastic policies in continuous-time reinforcement learning
• Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence

This page was built for publication: Convergence analysis for entropy-regularized control problems: a probabilistic approach

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q7234359)