Regularity and stability of feedback relaxed controls
From MaRDI portal
Publication:3382776
Abstract: This paper proposes a relaxed control regularization with general exploration rewards to design robust feedback controls for multi-dimensional continuous-time stochastic exit time problems. We establish that the regularized control problem admits a H"{o}lder continuous feedback control, and demonstrate that both the value function and the feedback control of the regularized control problem are Lipschitz stable with respect to parameter perturbations. Moreover, we show that a pre-computed feedback relaxed control has a robust performance in a perturbed system, and derive a first-order sensitivity equation for both the value function and optimal feedback relaxed control. These stability results provide a theoretical justification for recent reinforcement learning heuristics that including an exploration reward in the optimization objective leads to more robust decision making. We finally prove first-order monotone convergence of the value functions for relaxed control problems with vanishing exploration parameters, which subsequently enables us to construct the pure exploitation strategy of the original control problem based on the feedback relaxed controls.
Recommendations
- Reinforcement Learning for Linear-Convex Models with Jumps via Stability Analysis of Feedback Controls
- Patchy feedbacks for stabilization and optimal control: general theory and robustness properties
- Exploratory HJB equations and their convergence
- Reinforcement learning for exploratory linear-quadratic two-person zero-sum stochastic differential games
- Robust reinforcement learning control with static and dynamic stability
Cites work
- A neural network-based policy iteration algorithm with global \(H^2\)-superlinear convergence for stochastic games on domains
- A non-interior continuation method for generalized linear complementarity problems
- A smoothing-out technique for min—max optimization
- A strong comparison result for the bellman equation arising in stochastic exit time control problems and its applications
- Adapted Wasserstein distances and stability in mathematical finance
- All adapted topologies are equal
- Continuity of utility maximization under weak convergence
- Continuous‐time mean–variance portfolio selection: A reinforcement learning framework
- Controlled Markov processes and viscosity solutions
- Convergence of Dynamic Programming Models
- Convex Analysis
- Discontinuous Galerkin finite element approximation of Hamilton-Jacobi-Bellman equations with Cordes coefficients
- Discontinuous Galerkin finite element methods for time-dependent Hamilton-Jacobi-Bellman equations with Cordes coefficients
- Elliptic Partial Differential Equations of Second Order
- Existence and uniqueness theorems for solutions of McKean-Vlasov stochastic equations
- Extended weak convergence and utility maximisation with proportional transaction costs
- Generalized Hamilton-Jacobi-Bellman equations with Dirichlet boundary condition and stochastic exit time optimal control problem
- scientific article; zbMATH DE number 444922 (Why is no real title available?)
- scientific article; zbMATH DE number 3720745 (Why is no real title available?)
- scientific article; zbMATH DE number 3472400 (Why is no real title available?)
- scientific article; zbMATH DE number 1306981 (Why is no real title available?)
- scientific article; zbMATH DE number 1325009 (Why is no real title available?)
- scientific article; zbMATH DE number 1321699 (Why is no real title available?)
- scientific article; zbMATH DE number 1161554 (Why is no real title available?)
- scientific article; zbMATH DE number 7307478 (Why is no real title available?)
- Infinite dimensional analysis. A hitchhiker's guide.
- Linear and nonlinear functional analysis with applications. With 401 problems and 52 figures
- Proto-derivative formulas for basic subgradient mappings in mathematical programming
- Recursive approximation of the high dimensional max function
- Reinforcement learning. An introduction
- Robustness to incorrect system models in stochastic control
- Smoothing methods for convex inequalities and linear complementarity problems
- Some Convergence Results for Howard's Algorithm
- Uniqueness to elliptic and parabolic Hamilton-Jacobi-Bellman equations with non-smooth boundary
Cited in
(15)- Exploratory HJB equations and their convergence
- Reinforcement Learning for Linear-Convex Models with Jumps via Stability Analysis of Feedback Controls
- Optimal Scheduling of Entropy Regularizer for Continuous-Time Linear-Quadratic Reinforcement Learning
- Gradient flows for regularized stochastic control problems
- Optimal harvesting policy for biological resources with uncertain heterogeneity for application in fisheries management
- Near optimality of Lipschitz and smooth policies in controlled diffusions
- Continuous time reinforcement learning: a random measure approach
- Mirror descent for stochastic control problems with measure-valued controls
- Entropy annealing for policy mirror descent in continuous time and space
- Backward stochastic control system with entropy regularization
- Convergence of policy iteration for entropy-regularized stochastic control problems
- Convergence analysis for entropy-regularized control problems: a probabilistic approach
- Robustness of optimal controlled diffusions with near-Brownian noise via rough paths theory
- Feedback Cycles in Exploratory Equilibria
- Value-aware estimation of diffusion systems for continuous-time reinforcement learning: surrogate loss and financial applications
This page was built for publication: Regularity and stability of feedback relaxed controls
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3382776)
