Model-based reinforcement learning for approximate optimal regulation
From MaRDI portal
Abstract: In deterministic systems, reinforcement learning-based online approximate optimal control methods typically require a restrictive persistence of excitation (PE) condition for convergence. This paper presents a concurrent learning-based solution to the online approximate optimal regulation problem that eliminates the need for PE. The development is based on the observation that given a model of the system, the Bellman error, which quantifies the deviation of the system Hamiltonian from the optimal Hamiltonian, can be evaluated at any point in the state space. Further, a concurrent learning-based parameter identifier is developed to compensate for parametric uncertainty in the plant dynamics. Uniformly ultimately bounded (UUB) convergence of the system states to the origin, and UUB convergence of the developed policy to the optimal policy are established using a Lyapunov-based analysis, and simulations are performed to demonstrate the performance of the developed controller.
Recommendations
- Efficient model-based reinforcement learning for approximate online optimal control
- Reinforcement learning for optimal feedback control. A Lyapunov-based approach
- Reinforcement learning-based direct adaptive optimal control of JLQ model
- Mixed density methods for approximate dynamic programming
- A reinforcement learning-based scheme for direct adaptive optimal control of linear stochastic systems
Cites work
- A new adaptive law for robust adaptation without persistent excitation
- A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems
- A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems
- Adaptive dynamic programming for control. Algorithms and stability
- Adaptive Optimal Feedback Control with Learned Internal Dynamics Models
- Concurrent learning adaptive control of linear systems with exponentially convergent bounds
- scientific article; zbMATH DE number 4017400 (Why is no real title available?)
- scientific article; zbMATH DE number 47864 (Why is no real title available?)
- scientific article; zbMATH DE number 47865 (Why is no real title available?)
- Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems
- Model-free Q-learning designs for linear discrete-time zero-sum games with application to H^ control
- Multi-agent differential graphical games: online adaptive learning solution for synchronization with optimality
- Multi-player non-zero-sum games: online adaptive learning solution of coupled Hamilton-Jacobi equations
- Nonlinear control of engineering systems. A Lyapunov-based approach.
- OnActor-Critic Algorithms
- Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem
- Optimal control of unknown affine nonlinear discrete-time systems using offline-trained neural networks with proof of convergence
- Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning
- Pseudospectral methods for solving infinite-horizon optimal control problems
- Reinforcement \(Q\)-learning for optimal tracking control of linear discrete-time systems with unknown dynamics
- Robust adaptive LQ control schemes
- Robust Identification-Based State Derivative Estimation for Nonlinear Systems
Cited in
(39)- Optimal scheduling for reference tracking or state regulation using reinforcement learning
- General value iteration based single network approach for constrained optimal controller design of partially-unknown continuous-time nonlinear systems
- A model for system uncertainty in reinforcement learning
- Data-based reinforcement learning approximate optimal control for an uncertain nonlinear system with control effectiveness faults
- Adaptive neural finite-time control for space circumnavigation mission with uncertain input constraints
- Reinforcement learning for distributed control and multi-player games
- Mixed density methods for approximate dynamic programming
- Safe exploration in model-based reinforcement learning using control barrier functions
- Fixed-time estimation of parameters for non-persistent excitation
- Finite-horizon optimal tracking guidance for aircraft based on approximate dynamic programming
- Quantized adaptive tracking control for nonlinear systems with actuator backlash compensation
- Reduced-dimensional reinforcement learning control using singular perturbation approximations
- Optimal control of port-Hamiltonian systems: a continuous-time learning approach
- Approximate optimal influence over an agent through an uncertain interaction dynamic
- On exponentially convergent parameter estimation with lack of persistency of excitation
- Temporal logic guided safe model-based reinforcement learning: a hybrid systems approach
- Efficient model-based reinforcement learning for approximate online optimal control
- Multiple Model-Based Reinforcement Learning
- scientific article; zbMATH DE number 6795315 (Why is no real title available?)
- Recursive estimation in piecewise affine systems using parameter identifiers and concurrent learning
- Adaptive critic optimization to decentralized event‐triggered control of continuous‐time nonlinear interconnected systems
- Learning‐based super‐twisting sliding‐mode control for space circumnavigation mission with suboptimal reaching under input constraints
- Pretty darn good control: when are approximate solutions better than approximate models
- Omega-Regular Objectives in Model-Free Reinforcement Learning
- Composite adaptive neural tracking control of uncertain strict‐feedback systems
- Model-based reinforcement learning for approximate optimal control with temporal logic specifications
- Composite adaptive control of uncertain Euler-Lagrange systems with parameter convergence without PE condition
- Safe adaptive output-feedback optimal control of a class of linear systems
- Event-triggered-based integral reinforcement learning output feedback optimal control for partially unknown constrained-input nonlinear systems
- A multi-event-triggered control framework for nonzero-sum differential games of linear systems
- Control oriented reinforcement learning: a survey of recent progress and applications
- An adaptive linear quadratic tracker design for continuous – time systems with completely unknown dynamics
- Adaptive dynamic programming for robust neural control of unknown continuous-time non-linear systems
- Lyapunov-based adaptive deep system identification for approximate dynamic programming
- Lagrangian-based online safe reinforcement learning for state-constrained systems
- Synthesizing interacting model-based optimal control and model-free learning approaches for nonlinear systems
- Online Gaussian process learning based adaptive safe actor-critic control for continuous-time nonlinear systems
- Safe output-feedback adaptive optimal control of input-constrained control-affine nonlinear systems
- Model-based policy gradients with parameter-based exploration by least-squares conditional density estimation
This page was built for publication: Model-based reinforcement learning for approximate optimal regulation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q899267)
