Model-based reinforcement learning for approximate optimal regulation
From MaRDI portal
Publication:899267
Abstract: In deterministic systems, reinforcement learning-based online approximate optimal control methods typically require a restrictive persistence of excitation (PE) condition for convergence. This paper presents a concurrent learning-based solution to the online approximate optimal regulation problem that eliminates the need for PE. The development is based on the observation that given a model of the system, the Bellman error, which quantifies the deviation of the system Hamiltonian from the optimal Hamiltonian, can be evaluated at any point in the state space. Further, a concurrent learning-based parameter identifier is developed to compensate for parametric uncertainty in the plant dynamics. Uniformly ultimately bounded (UUB) convergence of the system states to the origin, and UUB convergence of the developed policy to the optimal policy are established using a Lyapunov-based analysis, and simulations are performed to demonstrate the performance of the developed controller.
Recommendations
- Efficient model-based reinforcement learning for approximate online optimal control
- Reinforcement learning for optimal feedback control. A Lyapunov-based approach
- Reinforcement learning-based direct adaptive optimal control of JLQ model
- Mixed density methods for approximate dynamic programming
- A reinforcement learning-based scheme for direct adaptive optimal control of linear stochastic systems
Cites work
- scientific article; zbMATH DE number 4017400 (Why is no real title available?)
- scientific article; zbMATH DE number 47864 (Why is no real title available?)
- scientific article; zbMATH DE number 47865 (Why is no real title available?)
- A new adaptive law for robust adaptation without persistent excitation
- A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems
- A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems
- Adaptive Optimal Feedback Control with Learned Internal Dynamics Models
- Adaptive dynamic programming for control. Algorithms and stability
- Concurrent learning adaptive control of linear systems with exponentially convergent bounds
- Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems
- Model-free \(Q\)-learning designs for linear discrete-time zero-sum games with application to \(H^\infty\) control
- Multi-agent differential graphical games: online adaptive learning solution for synchronization with optimality
- Multi-player non-zero-sum games: online adaptive learning solution of coupled Hamilton-Jacobi equations
- Nonlinear control of engineering systems. A Lyapunov-based approach.
- OnActor-Critic Algorithms
- Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem
- Optimal control of unknown affine nonlinear discrete-time systems using offline-trained neural networks with proof of convergence
- Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning
- Pseudospectral methods for solving infinite-horizon optimal control problems
- Reinforcement \(Q\)-learning for optimal tracking control of linear discrete-time systems with unknown dynamics
- Robust Identification-Based State Derivative Estimation for Nonlinear Systems
- Robust adaptive LQ control schemes
Cited in
(30)- Mixed density methods for approximate dynamic programming
- Reinforcement learning for distributed control and multi-player games
- scientific article; zbMATH DE number 6795315 (Why is no real title available?)
- A model for system uncertainty in reinforcement learning
- Efficient model-based reinforcement learning for approximate online optimal control
- General value iteration based single network approach for constrained optimal controller design of partially-unknown continuous-time nonlinear systems
- Multiple Model-Based Reinforcement Learning
- Adaptive critic optimization to decentralized event‐triggered control of continuous‐time nonlinear interconnected systems
- Model-based policy gradients with parameter-based exploration by least-squares conditional density estimation
- Learning‐based super‐twisting sliding‐mode control for space circumnavigation mission with suboptimal reaching under input constraints
- Finite-horizon optimal tracking guidance for aircraft based on approximate dynamic programming
- Adaptive neural finite-time control for space circumnavigation mission with uncertain input constraints
- Approximate optimal influence over an agent through an uncertain interaction dynamic
- On exponentially convergent parameter estimation with lack of persistency of excitation
- Model-based reinforcement learning for approximate optimal control with temporal logic specifications
- Event-triggered-based integral reinforcement learning output feedback optimal control for partially unknown constrained-input nonlinear systems
- Composite adaptive neural tracking control of uncertain strict‐feedback systems
- Data-based reinforcement learning approximate optimal control for an uncertain nonlinear system with control effectiveness faults
- Temporal logic guided safe model-based reinforcement learning: a hybrid systems approach
- Optimal scheduling for reference tracking or state regulation using reinforcement learning
- Optimal control of port-Hamiltonian systems: a continuous-time learning approach
- Safe adaptive output-feedback optimal control of a class of linear systems
- Composite adaptive control of uncertain Euler-Lagrange systems with parameter convergence without PE condition
- Safe exploration in model-based reinforcement learning using control barrier functions
- Omega-Regular Objectives in Model-Free Reinforcement Learning
- Pretty darn good control: when are approximate solutions better than approximate models
- Quantized adaptive tracking control for nonlinear systems with actuator backlash compensation
- Recursive estimation in piecewise affine systems using parameter identifiers and concurrent learning
- Reduced-dimensional reinforcement learning control using singular perturbation approximations
- Fixed-time estimation of parameters for non-persistent excitation
This page was built for publication: Model-based reinforcement learning for approximate optimal regulation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q899267)
