Model-based reinforcement learning for approximate optimal regulation
From MaRDI portal
Publication:899267
DOI10.1016/J.AUTOMATICA.2015.10.039zbMATH Open1329.93051arXiv1304.3477OpenAlexW2086975818MaRDI QIDQ899267FDOQ899267
Rushikesh Kamalapurkar, Patrick Walters, Warren E. Dixon
Publication date: 23 December 2015
Published in: Automatica (Search for Journal in Brave)
Abstract: In deterministic systems, reinforcement learning-based online approximate optimal control methods typically require a restrictive persistence of excitation (PE) condition for convergence. This paper presents a concurrent learning-based solution to the online approximate optimal regulation problem that eliminates the need for PE. The development is based on the observation that given a model of the system, the Bellman error, which quantifies the deviation of the system Hamiltonian from the optimal Hamiltonian, can be evaluated at any point in the state space. Further, a concurrent learning-based parameter identifier is developed to compensate for parametric uncertainty in the plant dynamics. Uniformly ultimately bounded (UUB) convergence of the system states to the origin, and UUB convergence of the developed policy to the optimal policy are established using a Lyapunov-based analysis, and simulations are performed to demonstrate the performance of the developed controller.
Full work available at URL: https://arxiv.org/abs/1304.3477
Recommendations
- Efficient model-based reinforcement learning for approximate online optimal control
- Reinforcement learning for optimal feedback control. A Lyapunov-based approach
- Reinforcement learning-based direct adaptive optimal control of JLQ model
- Mixed density methods for approximate dynamic programming
- A reinforcement learning-based scheme for direct adaptive optimal control of linear stochastic systems
system identificationadaptive controldata-based controlmodel-based reinforcement learningconcurrent learningsimulated experience
Learning and adaptive systems in artificial intelligence (68T05) Applications of optimal control and differential games (49N90) System identification (93B30) Adaptive control/observation systems (93C40)
Cites Work
- Title not available (Why is that?)
- Multi-agent differential graphical games: online adaptive learning solution for synchronization with optimality
- Optimal control of unknown affine nonlinear discrete-time systems using offline-trained neural networks with proof of convergence
- Title not available (Why is that?)
- Adaptive dynamic programming for control. Algorithms and stability
- OnActor-Critic Algorithms
- Nonlinear control of engineering systems. A Lyapunov-based approach.
- Robust adaptive LQ control schemes
- Pseudospectral methods for solving infinite-horizon optimal control problems
- Title not available (Why is that?)
- Reinforcement \(Q\)-learning for optimal tracking control of linear discrete-time systems with unknown dynamics
- Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem
- A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems
- Concurrent learning adaptive control of linear systems with exponentially convergent bounds
- Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems
- A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems
- A new adaptive law for robust adaptation without persistent excitation
- Multi-player non-zero-sum games: online adaptive learning solution of coupled Hamilton-Jacobi equations
- Model-free \(Q\)-learning designs for linear discrete-time zero-sum games with application to \(H^\infty\) control
- Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning
- Robust Identification-Based State Derivative Estimation for Nonlinear Systems
- Adaptive Optimal Feedback Control with Learned Internal Dynamics Models
Cited In (28)
- On exponentially convergent parameter estimation with lack of persistency of excitation
- Data-based reinforcement learning approximate optimal control for an uncertain nonlinear system with control effectiveness faults
- Safe exploration in model-based reinforcement learning using control barrier functions
- Composite adaptive control of uncertain Euler-Lagrange systems with parameter convergence without PE condition
- Omega-Regular Objectives in Model-Free Reinforcement Learning
- A model for system uncertainty in reinforcement learning
- Composite adaptive neural tracking control of uncertain strict‐feedback systems
- Title not available (Why is that?)
- Safe adaptive output-feedback optimal control of a class of linear systems
- Approximate optimal influence over an agent through an uncertain interaction dynamic
- Event-triggered-based integral reinforcement learning output feedback optimal control for partially unknown constrained-input nonlinear systems
- Multiple Model-Based Reinforcement Learning
- Temporal logic guided safe model-based reinforcement learning: a hybrid systems approach
- Finite-horizon optimal tracking guidance for aircraft based on approximate dynamic programming
- Learning‐based super‐twisting sliding‐mode control for space circumnavigation mission with suboptimal reaching under input constraints
- Recursive estimation in piecewise affine systems using parameter identifiers and concurrent learning
- General value iteration based single network approach for constrained optimal controller design of partially-unknown continuous-time nonlinear systems
- Quantized adaptive tracking control for nonlinear systems with actuator backlash compensation
- Pretty darn good control: when are approximate solutions better than approximate models
- Fixed-time estimation of parameters for non-persistent excitation
- Model-based reinforcement learning for approximate optimal control with temporal logic specifications
- Efficient model-based reinforcement learning for approximate online optimal control
- Optimal scheduling for reference tracking or state regulation using reinforcement learning
- Optimal control of port-Hamiltonian systems: a continuous-time learning approach
- Adaptive neural finite-time control for space circumnavigation mission with uncertain input constraints
- Mixed density methods for approximate dynamic programming
- Reinforcement learning for distributed control and multi-player games
- Adaptive critic optimization to decentralized event‐triggered control of continuous‐time nonlinear interconnected systems
Uses Software
This page was built for publication: Model-based reinforcement learning for approximate optimal regulation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q899267)
