Adaptive dynamic programming for model-free tracking of trajectories with time-varying parameters
From MaRDI portal
Abstract: In order to autonomously learn to control unknown systems optimally w.r.t. an objective function, Adaptive Dynamic Programming (ADP) is well-suited to adapt controllers based on experience from interaction with the system. In recent years, many researchers focused on the tracking case, where the aim is to follow a desired trajectory. So far, ADP tracking controllers assume that the reference trajectory follows time-invariant exo-system dynamics-an assumption that does not hold for many applications. In order to overcome this limitation, we propose a new Q-function which explicitly incorporates a parametrized approximation of the reference trajectory. This allows to learn to track a general class of trajectories by means of ADP. Once our Q-function has been learned, the associated controller copes with time-varying reference trajectories without need of further training and independent of exo-system dynamics. After proposing our general model-free off-policy tracking method, we provide analysis of the important special case of linear quadratic tracking. We conclude our paper with an example which demonstrates that our new method successfully learns the optimal tracking controller and outperforms existing approaches in terms of tracking error and cost.
Recommendations
- A novel adaptive dynamic programming based on tracking error for nonlinear discrete-time systems
- Online optimal tracking control of continuous-time linear systems with unknown dynamics by using adaptive dynamic programming
- Robust approximate optimal tracking control of time-varying trajectory for nonlinear affine systems
- Approximate optimal trajectory tracking for continuous-time nonlinear systems
- Multiple model adaptive tracking control based on adaptive dynamic programming
Cites work
- 10.1162/1532443041827907
- \({\mathcal Q}\)-learning
- A parallel distributed supervision strategy for multi-agent networked systems
- scientific article; zbMATH DE number 5562427 (Why is no real title available?)
- scientific article; zbMATH DE number 1321699 (Why is no real title available?)
- scientific article; zbMATH DE number 3388902 (Why is no real title available?)
- Linear Quadratic Tracking Control of Partially-Unknown Continuous-Time Systems Using Reinforcement Learning
- Optimal control
- Reinforcement \(Q\)-learning for optimal tracking control of linear discrete-time systems with unknown dynamics
- Reinforcement learning. An introduction
- Stabilization of strictly dissipative discrete time systems with discounted optimal control
Cited in
(9)- Learning output reference model tracking for higher-order nonlinear systems with unknown dynamics
- Robust approximate optimal tracking control of time-varying trajectory for nonlinear affine systems
- Event‐trigger‐based approximate optimal control of modular robot manipulators using zero‐sum game
- Optimal event‐triggered control for C‐T system with asymmetric constraints based on dual heuristic dynamic programing structure
- Assured learning-enabled autonomy: a metacognitive reinforcement learning framework
- Eligibility traces and forgetting factor in recursive least-squares-based temporal difference
- A new Q-function structure for model-free adaptive optimal tracking control with asymmetric constrained inputs
- Data-driven based optimal output feedback control with low computation cost
- Output feedback control of anti-linear systems using adaptive dynamic programming
This page was built for publication: Adaptive dynamic programming for model-free tracking of trajectories with time-varying parameters
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5003423)
